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CHAPTER I 



Executive Summary \ 



Background 

The central purpose of this study was to evaluate the implementation of 
the Stal lings Classroom Management Staff Development Demonstration Project 
in Putnam County Schools, West Virginia, during school Vear 1980-1981. The 
Stal lings Classroom Management Staff Development Program\is a product of over 
ten years of research and development in the area of teacher effectiveness 
and teacher training research. The purpose of the program^ is to increase 
student achievement in basic skills through the utilization of research- 
based, systematic change in teachers' classroom behaviors, especially as 
these teaching behaviors relate to the management of classroom instruction 
fime and the organization of classroom activities. The demonstration 
project was implemented at two of the four high schools in Putnam County. 
Both schools' language arts faculties were involved (N=ll). \ 



of Schools. and the evaluator. The four objectives were: (1) to conduct a 
process and product evaluation of the implementation of the Stal lings Class- 
room Management Staff Development Demonstration Project in Putnam County, 
West Virginia; (2) to utilize techniques and to administer a set ofyinstru- 
ments, in the- evaluation and. assess-. their util ity/results for future ^evalua.7 — 
tions in the county;' (3) to make data-based recommendations regardingf uture 
implementation of the project and communicate same to the Superintendent; 
and (4) to share the results of the -evaluation with educators at- all levels. 

» 

Evaluation Methods \ \ - 

This evaluation of the Starlings staff development program in Putnam 
County Schools util-ized both process and product evaluation methods. Process 
evaluation focused on project teachers' expressed reactions/feelings as the 
demonstration project unfolded, teachers' concerns about\the innovation, and 
evaluative comments made by the involved teachers during taped interviews at 
the conclusion of the project. Product evaluation focusea\ on the pre-post- 
test changes in teachers' perceived responsibility for student, achievement > 
in the classroom, ratings on the project teachers' Levels of Use of the 
innovation, and the demonstration project teachers' pre-posttest intervention 
observation changes in "correct implementation" (as defined by the program) 
of specific classroom teaching behaviors. Also s selected administrators were 
interviewed at the conclusion of the demonstration project to determine their 
reaction to the project. ^ 

Process Evaluation/Results 

On a scale of "+3" to "-3," the teachers' expressed reactions/feelings 
moved upwarl from <jn initial +0.5 through -0.1 and ended up at +1.6. The 
lowest re alt ion/fee ling score was for the point .In, time, gust after the- first 
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teacher training workshop in which they received their individual teacher 
behavior profile charts. For both schools 1 teachers combined, the most 
intense concern was Informational at the 88th percentile. Across the 
schools, only Stage Two--Perso.nal--produced a significant difference with 
that difference being in two specific items. A total of 685 evaluative 
comments were gleaned from the eleven teachers' interview tapes. Of those, 
49Q were coded as positive Gomments jnd 195 were coded as negative. 

Product Evaluation Results 

There was a significant difference in the positive direction for the 
teachers^ expressed responsibility for student achtevemept before and after 
the staff development demonstration project. : PQSjt -project Levels of Use * 
ratings showed the group to include eight innovation users and three non- 
users. A total of 19,885 teache*-f bcused 'interactions were coded from the 
pre- and post-observations. It was determined that the teachers- 1 pre-post- 
observations of correct implementations (as define^ by the program) of 
specific teaching behaviors changed significantly in the positive direction. 
Jhe mean number of pre-observation correct implementations of recommended 
teaching behaviors (N=45) was 21.73 while the post-observation mean was ' 
io.82, an increase of nine correct implementations. A new measure, the 
Teaching Behaviors Change Index (TBCI) was conceited by this evaluation. Ten 
of\the eleven teachers* TBCI values were 'positive. Generally, the two high 
school* principals, the teacher trainer, and the Superintendent of Schools 
voi&ed positive reactions to- the demonstration project during interviews. 
The\eacher trainer provided seven - specif ic recommendations for improvement 
in me program's processes.- — — ~~~~ ' 

Recommendations « 

fles\lts >df t^is evaluation show that the Stallings Classroom Management 
Staff Development Project implemented in Putnam County Schools in 1980-1981 
was a success. Nothing was discovered in the evaluation of the demonstra- 
tion project to discourage its continued use and, resources permitting, its 
expansion A Some recommendations for slight refinement in the program 
include: \l) expand the classroom observations to five class periods per * 
set, (2) spread these five class observations over a two-week time period, 
(3) d& not distribute the teacher behavior profile charts until the second 
teacher training workshop, and (4) expand the number of workshops.to six to 
accommodate the previous recommendation. It was recommended that a planned 
and coordinated information program about the staff development project be 
designed and implemented including a' program/project logo, an informational 
brochure, awareness meet ings , newspaper articles, and other activities. It 
was recormended\that Putnam County Schools seek self-sufficiency with the 
program through training ajocal "apprentice" and purchasing the optical 
scanning and date\ analyses programs. It was recommended: (I) to monitor 
continuously the staff development program and including a comparison group 
of teachers iq the\design, v (2) to deyelop*a criterion-*eferen£ed, achievement 
data base, (3) to continue to report and display the Teaching Behavior 
Change Index y&lues; an,d (4) to communicate the result? of this evaluation 
to other educators at all levels through various means^ ; . 



* CHAPTER II V \i 

■ ■ : \ 

Introduction ) 

» 1 

Purpose »•« 

The purpose of this investigation was to conduct an evaluation of the 
Stall ings Classroom Management Staff Development Demonstration Project imple- 
mented in the Putnam County Schools, West Virginia during the 1980-81 school 
year. Requested by the Superintendent of Schools, the underlying assumption 
of the, evaluation was that the administrator^, and teachers of the demonstra- 
tion project could .prof it from objective analyses of the project's implemen- 
tation*™ their schools. Administrators could revise, adjust, or terminate 
the project based on data from an evaluation of its trial installation, 
rather than rely on intuition. It was felt that teachers could profit from 
an outside party's assessment of T>oth the processes they went through and 
the products of their efforts. 

Objectives 

Four main evaluation objectives were agreed upon by the Superintendent 
of Schools and the evaluator. Agreement oil these objectives was reached 
early and then they formed the basis for the design and conduct of the evalu- 
ation. The first objective was to conduct a process and product evaluation 
of the implementation of the Stallings Classroom Marragment Staff Development 
Demonstration Project in Putnam County, West Virginia. The second objective 
was to utilize techniques and to administer a set of instruments irt the 
evaluation and assess their utility/results for future evaluations of similar 
programs in the county. The evaluation's third objective was to* make data- 
based recommendations regarding future implementation of the project and 



communicate same to the Sup^uit^ The fourth objective of the study 

was to share the results of the evaluation with other educators. 

N> J-' 

Background of the Stallings Classroom 
Management Staff Development Program 

The Stall ings Classrqom Management Staff Development Program for teachers, 
is- the result of continuing research and.devel opment conducted by Jane A. 
Stall ings. Much of the research was conducted under various programs and 
grants received by Stallings and her associates during a ten-year period at 
SRI International, Menlo Park, California. The work on disseminating, 
expanding, and refining the staff development program continues under 
Stallings 1 direction at the Teaching and Learning Institute, a nonprofit 
corporation centered in Mountain View, California. 

The framework and background for the Stal 1 ing||Classroom Management Staff 
Development Program rests in the teacher effectiveness research of the late 
1960's and the 1970*5 . Stallings played a continuing role in this strand of 
educational research. The whole era of the process-prod tt uct studies (class- 
room pf.ocesses--educatibnal products) was organised, into several stages-- 
although the stages were not finely defined in time and/or pre-established £ 
categories. In the first stage, several major correlational studies were 
conducted to describe and examine closely actual classroom practices and to 
relate these documented practices upon student achievement outcomes in the 
basic £kf 11' areas (Stallings and Kaskowitz, 1974; Berliner and Tiku^off, 
1979; McDonald and Elias, 1976; Brophy and Evertson, 1976; Soar and Soar, 
1972; and Good and Grouws, 1977) . These studies were conducted at the 
elementary school level. This stage of teacher effectiveness research 
produced dependable knowledge of the relationships between teachers 1 
behaviors and student outcomes as measured by standardized achievement 



instruments. The direct instruction model and the concept of students' time- 
on-task are the most widely known outcomes of these early studies (Rosen- % 
shine, 1979). , 

The second stage in the teacher effectiveness line of research which 
serves as the basis for the Stal lings staff development program was the 
application of the design, methodologies, and techniques of the first stage 
to secondary-level classrooms. Here, the results were similar to the elemen- 
tary level studies regarding the direct instruction model (Stal lings and ; \ 

r 

others, 1978; Anderson, Evertson, and ,Brophy, 1979). In some of these 
' studies, methodological improvements were realized. The time-on-task concept 
was refined to include further operational concepts and definitions. 
Stal lings (1980) refined-the time-on-task activities by dividing them into 
interactive instructional activities *nd noninteract ive instructional acti- 
vities. 

In the third stage a series of field studies or quasi-experiments were 
conducted by the same sets of conceptually similar but operationally separate 
teams of researchers. The common purpose of these studies was to test the 
applicability of the previous research results as major components in 
teachers 1 inservice training. Generally, these studies (1) produced statis- 
tically significant changes in kfl^ teaching behaviors, and (2) favored the 
treatment teachers over the comparison teachers in producing student learning 
gains on standardized achievement tests (Stal lings, Needels, and Stayrook, 
1979; Anderson, Evertson, and Brophy, 1979; Crawford and others, 1978; and 
Good and .Grouws, 1977). Results from these field studies or quasi-experi- 
ments reinforced the previous research in identifying correlations between 
classroom processes such as specific teaching behaviors and student 

ERIC 



achievement outcomes. Thus, more evidence of linkages between classroom 
teaching behaviors and student achievement was gained. 

We are now in the fourth stage in teacher effectiveness research. How- 
ever, by stating that the research is in the fourth stage now is not to s.ay 
that the other stages have been completed— there are no fine lines separating 
the various stages and teacher effects research may never be complete. This 
fourth stage may be thought of as the dissemination stage. The purpose at 

■s 

this stage, is to make the results, of tested aspects of teacher effectiveness 
research available to educational practitioners so that they might utilize 
the results of educational research in improving instructional practices in 
schools. For example, teacher effectiveness researchers at The University 
of Texas R&D Center for Teacher Education, Austin, Texas, have disseminated 
their work on classroom organization and management. They have published a 
series of research reports (1981), a teacher training videotape, and, a 
teacher training manual at the elementary level (Evertson, Emmer, Clements, 
Sanford, Worsham, and Williams, 1981). Further, they have responded to 
requests from the field by conducting numerous dissemination workshops/ 
seminars across the country (Sanford, Clements, and Emmer, 1981). 

Stal lings took a slightly different route to disseminating her Class- 
room Management Staff Development Program. In 1979 she submitted to the ^ 
Joint Dissemination Review Panel (JDRP) to have the program val-idated based 
upon the rigorous criteria established by the JDRP (Stallings, 1979)./ At 
that time the program was labeled "The Process of Teaching Basic Reading* 
Skills in Secondary Schools." The submission was approved by JDRP. As a 
function of JDRP approval, the program became eligible for National Diffu- 
sion Network (NDN) funds to become a developer/demonstrator (D/D) project. 
Being JDRP' approved/ the Stallings program was listed in the catalog of 



approved projects called Educational Programs That Work (1980). Interested 
educational practitioners contact the program developers and the D/D can 
provide more awareness materials and/or inperson presentations. 

To facilitate actual replications of the program in other settings, 
Stall fngs designed an "apprenticeship system." Stal lings and her associates 
trained carefully selected candidates in both the classroom observation 
system and the teacher training system. These "apprentices" then returned 
to their home agencies to install the model. The apprentice's work was 
monitored by Stal lings through tape" recorded sessions, written reports on 
the teacher training sessions, and other means. After the system had been 
in place for one year and judged satisfactory, the apprentice was "approved" 
and eligible to train other, new apprentices in other agencies. Thus, the « 
Stal lings system expands with carefully selected and approved teacher 
trainers. A description ancJ assessment of the implementation of the. 
Stallings Classroom Management Staff Development Program in ^n urban school 
system is provided by Ghory and Cash (1981). 

Description of the Stallings Classroom v * 

Management Staff Development Program 

The Stallings Classroom Management Staff Development Program is a product 
of over ten years of research and development in. the area of teacher effec- 
tiveness and teacher training research. A rather comprehensive "chronology of 
events in this- continuing research, up until 1977,, was provided in Learning, 
to Look: A Handbook on Classroom Observation and Teaching Models (Stallings, 
1977). More recent references have been cited earlier in tWs report. 

The central purpose ofthe Stal 1 ings tlassroom Management Staff Develop- 
ment Program is to increase student achievement in basic skills through the 
utilization of research-bas.ed, systematic change in teachers 1 classroom 



behaviors., especially as th^se teaching behaviors relate to the management of 

* ' .. • 1 • . : -' V . ■ 

tlassroom instruction time and the organization of classroom activities*. 

\ Stall i^gs (1981) provides a capsule description of the two key glements 

in her cl Jssroom management' staff development program: 

Every staff development model contains a curriculum arid a 
delivery system. Curriculum means the content and delivery * ■».. 
means where, when, how, and number of participants. A 
good content with poor delivery, or vice versa, is not 
likely to be effective in bringing about .change in teacher 
behavior. . 

The goal of'the Teaching and Learning Institute's training 
program^ is to help teachers learn to manage their classrdom 
time effectively. The curriculum is based upon research 
findings. The delivery system is .personal i zed instruction 

and interactive sjnal 1 group problem solving (page 6). 

✓ 

Each element shall be discussed in turn. 

The content of the Stallings Classroom Management Staff Development Pra- 
qram is derived from research funded over a period pf years by the National 
Institute of Education (NIE). The research-based content of the -Sjal lings 
program involves two essential processes. The first process is the system- 
atic collection of classroom data through the use of a multi-component, low- 
inference observation system. The Secondary Observation Instrument (SOI) is 
the heart of the data collection system (Stallings and Needels, 1981). The 
SOI provides a convenient method for recording objective data on the interac- 
tions within classrooms. These interactions are teacher-based primarily' but 
do include student grouping patterns and teacher aide activities. 

The Identification and Classroom 'Information Component is the first part 
of the SOI. It is coded only once during a class period. This component 
records identification information jriedessary for optical scanning and data 
processing. Information on. the teacher, grade level, class size, observer 
information, and the observation date are recorded. , 



The Classroom Snapshot is the second component of the SOI. Thfe Classroom 

i 

Snapshot is a one page form which records the mental picture of the total 
classroom with, all its students and adults, what they are doing, apd with what 
materials at one point in time. It is analagous to a "Polaroid picture" only 
in data coding terms. The Snapshot, as;it is commonly called, is completed 
five 'triiies during each classroom observation (class period) session. The Snap 
shojtrfrecords every person's whereabouts and involvement at that 6ne instant, 
r/. The third component of the SOI is the Five-Minute Interaction (FMI). The 
.VFMI records the verbal interactions and activities as they transpire in the 
/classroom during the timeframe" of this part -of the observation. FMI data are 
coded into the four columns of who, to whom, what, and how. In the FMI part 
of the SOI, the teacher is the focus of the coding,. Coded for five minutes 
following the completion of each Snapshot, the five FMIs yield approximately 
300 verbal interactions for each observation session. When assembled and 
combined in a regular pattern, the component parts of the SOI form a booklet. 

In the Sta'l lings Classroom Management Staff Development Program, class- 
room observations are conducted for three consecutive days in the same class 
period—selected ahead of time by the teacher as the "target" class. Thus, 
in addition to the identification data, a total of 15 Classroom Snapshots and 
approximately 900 verbal interactions are recorded per each class. 

Data analysis is the next step." The completed SOI data collection book- 
lets are mailed to a firm in Minnesota for optical scanning. A magnetic data 
tape is produced holding the SOIs 1 raw data. These computer tapes are 
forwarded to the Teaching and Learning Institute where the data analyses are 
performed. Under Stallings* direction, a series of data analyses programs 
were developed which records, analyzes, aggregates, and displays the SOI 
data. The results are computer-generated "profiles" of teacher behaviors. 

ij 
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One teacher profile is geared to the Snapshot variables while the second is 
geared to the FMI variables. The teacher behavior profile charts produced 
at this stage include the naming of 45 teaching variables such as: adult to 
individual student, student reading aloud, adult praise/support, total ■ 
practice drill, total silent reading, and all social qomments. The computer- 
generated teacher profile charts graphically depict the relative position of 
the subject teacher in comparison to the mean of the total group observed 
and simple recommendations of "More" or "Less" are made. These' "More" and 
"Less" recommendations to the teacher are based on research findings 
regarding student achievement outcomes derived from researcl^y^elding four 
levels of student achievement gain/no-gain as measured by a norm-referenced 
instrument. These teacher profiles and recommendations for teacher behavior 
changes are the content of the teacher training workshops which will be 
explained in the following paragraphs. 

The delivery system in the Stal lings Classroom Management Staff Develop- 
ment Program is "personalized instruction and interactive small group 
problem-solving" (Stal lings, 1981). Groups of teachers attend a series of 
workshops after the initial observations have been completed, data analyzed, 
and teacher profiles generated. Workshop sessions are conducted one week : 
apart. The workshop sessiop^ace interrelated but each one has its own topic 
and content.' The workshop session^ are process-oriented in that the small 
group of teachers (usually 5-7) are encouraged to offer, discuss, try, and 
provide feedback regarding new teaching behaviors, techniques, and instruc- 
tional activities. Workshop materials, including a packet developed by the 
Teaching and Learning Institute and additional materials selected by the 
teacher trainer, are prepared ahead of time, but the teacher trainer primar- 
ily acts as group facilitator. The focus of the workshops is to present 
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recommendations for teacher behaviors based on educational research findings; 
to identify, locate, and discuss these various teacher behaviors among the 
group members;, to^J^lp teachers to implement teaching changes in their class- 
room; ahd to provide feedback to individuals and the small group. 

In addition to more complete descriptions of the Stal lings Classroom 
Management Staff Development Program contained in the references cited 
previously, there is a companion volume to this report which describes the 
program in greater detail (Sullivan, Basile, and Higginbotham) . The 
companion narrative report describes the implementation of the Stal lings 
Classroom Management Staff Development Program in the Putnam County School's,, 
West Virginia. 
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CHAPTER III 

i 

i 

Evaluation Procedures 

The central purpose of this study was to evaluate the implementation of 
the Stall ings Classroom Management Staff Development Demonstration Project 
in Putnam County Schools, West Virginia. The previous chapter presented the 
..background and the description of I the Stall ings Classroom Management Staff 
Development Program in brief. The purpose of this chapter is to present the 
data collection procedures of the evaluation effort. 

The first objective of the demonstration project was to "Install, moni- 
tor, evaluate, and reconceptual ize a demonstration site in Putnam County for 
the Stalling* Classroom Management Staff Development Model" (project records) 
This evaluation addresses directly the "evaluate" part of the objective and, 
further, the recommendations provide input for the reconceptual ization por- 
tion of the objective. 

Evaluation Design ^- s - — 

This evaluation effort fits into the "decision-making" category of major 
models of evaluation as explicated by Worthen arid Sanders (1973). The major 
proponent and the classic reference on this model is Stuff lebeam and others 
(1971). In this model the evaluation is structured by the information needs 
of the decision-maker. In this case, the decision-maker was the Superinten- 
dent of Putnam County Schools, West Virginia. Therefore, the evaluator met 
with the Superintendent to develop a plan to supply information which would 
help him make decisions about the project. ^ 

The Putnam County demonstration project of the Stall ings Classroom 
*< Management Staff Development Program was just that--a one-time demonstration 
project. It was decided that the primary focus of the evaluation was to 
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study the effects of the implementation on the teachers involved. The secon- 
dary focus y*as the, demonstration sites* building principals and the project's 
teacher trainer (labeled the Stallings apprentice in previous sections, but 
switched now to the more accurate and descriptive term). v Discussions between 
the Superintendent of Schools and the evaluator determined that the most 
important information need rested in the area of the impact of their involve- 
ment in the project on the "subject '' teachers both in terms Of the. processes 
they went through and results of their involvement. In this sense, then* the 
evaluation was divided into a process evaluation and a product evaluation. 
Figure 1 is a graphic, of the evaluation design and its instruments. Note, 
though, that these two designations of process and product evaluation differ 
from the traditional use of these terms* in evaluation literature. To reca- 
pitulate, within this evaluation report, process evaluation shall mean 
assessing teachers* involvement in the demonstration project's various acti- 
vities while product evaluation will assess the impact of these activities 
upon the involved teachers. * 

» *■* , : . w ■ "•" 

Sam pi e^ . ... • ; v 

The demonstration project was implemented at two of the fogr high schools 
in Putnam County. The selected schools 1 language arts education faculties 
were asked to participate in the demonstration project as one element in 
their continuing professional development. The language arts education 
faculty at one high school numbered four teachers while the other school's 
language arts education faculty numbered seven^teachers. These 11 teachers 
were the subjects for this evaluation's data collection. Given the small 
number of demonstration project teachers (necessarily due to the delivery 
system of the program), it was decided to collect data from alTteachers. 
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Also, additional perceptions-type data were collected from the two school 
principals, the project's teacher tr<ainer, and the Superintendent of Schools.? 
Table 1 presents the demonstration project teacher s* background informa- 

tion. Project teachers included nine females and two males. Their ages 

\ ' * ' ? ' , • 

\ ranged from the 20-29 age category to- the' 50-59 age category although nearly 

\ ■ " . L 

\ 64 percent were in the 30-39 category/ The total number of years of teaching 

experience ranged from 2 to, J 3 years with fiour teachers (36%) having either 
four or five years of experience. The numbe^ of years the teachers had taught 
at their present schools ranged from one tojnlne although, interestingly 
\ enough, four teachers (36%) were completing thk first -year at their school; 1 

process Evaluation Instruments ' ' 

V Two instruments were employed specifically to col lee^ process evaluation 
dal^a while one instrument served to collect both process and product data. 
Eachy-shall be described in turn. „ fV 
: R^acl i on Survey . ' r -fe of the, teachers 1 reactions to the demon- 

stration project's various activities from start to finish were elicited by 

\ . ' ■ ' ' ' 

the locally-developed Reaction Survey. This Reaction Survey was administered 

at the conclusion of the project. The Reaction Survey was .a simple one-page 

\ ■ 

self-report form with the 12 spaces to be filled in with statements from the 

\ '. '. 

teachers. Each space included an important event phrase as a stimulus intern. 



Teachers were asked to respond to ttie stimulus items by writing in their 
reactions and/or feelings at that particular time. A copy of the Reaction^ 
Survey appears in Appendix A. In essence, the Reaction Survey was an instru- 
ment soliciting post^project reactions/feelings from participants about the 
\- major events in the sequence of project activities. The advantage of this 
technique was that the teachers had time to put all the various events and 
their reactions into a total perspective after the project was completed. 

ERIC O ^ 2u 
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Table 1 

Project leathers 1 Background Information (N*ll) 



Item 



Number 



Percent 



Sex: 

Male 
Female 

Aqe Category:* \ ^ 

20-29- 
30-39 
40-49 
50-59 ' 

Total Years Teaching: 3 

1 

2 
3 
4 
5 
6 

7 v 

8. 

9 

10 
13 

Number Years at 1 
Present School:" 



1 

2 



8 
9 



2 
9 

TT 



2 
7 
1 
1 

TT 



0 
1 
1 
2 
2 
0 
0 
1 
1 
1 
1 
0 

1 

TT 



4 

2 
2 



aMean =6.73 
b Mean =3.50 , 
c Does not equal 100 due to rounding 




18.2 
81.8 



18?>2 
63.6 
9.1 
,9.1 
T0TT0" 



0 

9.1 
§.1 

18.-2 
18.2 

0 

0 

9.1 
9.1 
9.1 

9.% 
0 

9.f 
lOOTF 



36.4 
18.2 
18.2^ 



18.2 
9.1 
100. 1^ 
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The teachers 1 Reaction Survey was developed by the evaluator in coopera- 
tion with the teacher trainer., A check of the demonstration project's 
calendar and events produced the "first cut" of significant events in the 
implementation of the project. This initial list was checked by the teacher 
trainer for accuracy. Consensus was reached between the two individual? on ( 
the final list of events used as stimulus items. Thus, the Reaction Survey 
possessed a degree of face validity (Issac and Michael, 1971). The utility 
of the Reaction Survey as a device. to gather reactions, over a\ period of - • 
time, to a'project's major events had been determined previously by Meehan*; 
and Basile (in press). Scoring the Reaction Survey consisted of ; ,the evalu- 
ator reading carefully the various prose descriptions for each teacher's 
instrument, mentally "feeling" the global range of reactions, then rereading 
each individual section and assigning a point value to it. The point values 
ranged from a "+3" or "Very Positive," through "0" denoting a neutral value, 
to a "-3" denoting a "Very Negative" reaction. The evaluator's assignment 
of point values to prose statements was checked by an independent evaluator. 
There was high initial ipterrater agreement between these two persons on most 
items. Then they disqussed each area of nonagreement until consensus was 
reached on each section of each teacher's Reaction Survey. The result was 
100% tnterrater agreement. However, no other reliability checks were com- 
pleted and readers should interpret the resultant data with this caution in 
mind. * 

Stages of Concern Questionnaire . The Concerns-Based Adoption Model 
(CBAM) is a conceptual framework for describing, assessing, and facilitating 
change in schools and colleges/. The foundation document in the CBAM frame- 
work is by Hall, Wallace, and Dossett (1973). Based on the earlier work of 
Frances Fuller (1969), the CBAM is a research-based system for studying 
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teacher changes in teWjs of the/ir concerns and behaviors, as they are 
involved in an educational innovation. Fuller proposed three phases of ' 

I 

concern: self, task, and impact. 

Stages of Concern (SoC) is one dimension of the CBAM (Hall, George, and 
Rutherford, 1979). These authors devoted a section of ,their report to iden- 
tifying and describing teachers 1 "concerns" (pages 4-5). < One paragraph 
particularly conveys the concept of concerns: 

The composite representation of the feelings, preoccupation, 
thought, apd cqnsideration given to a particular task is 
called concern*. ...the mental activity composed of ques- , 
tioning, analyzing, and reanalyzing, considering- alterna- * 
ative actions and reactions, and anticipating consequences 
/is concern. Ap aroused state of personal feelings and thought - 
about a demand as it is perceived is cdncern (page 5). ^ 

These writers also stated that it is the person's perceptions that stimu- 
late concerns—"not necessarily the reality of the situation" (page 5). 

Research conducted at the Research and Development Center for Teacher 
Education at The University of Texas at Austin over the past eight years has 
identified, assessed, confirmed, and documented the cpncerns dimension of the 
CBAM. CBAM researchers have monitored teachers 1 concerns in cross sectional 
and/or longitudinal studies of 11 different educational process and product 
innovations. Their research has confirmed that there are usually differen- 
tial degrees of arousal or concern with regard to the innovation. Indivi- 
duals„regard certain demands of the innovation as morejimportant „than others 
at a given time. Thus, "the degree of arpusal (intensity) of the different 
types of concerns will vary" (Hall, George, and Rutherford, 1977, page 5). 
Additionally, these researchers state that "there appears to be a predictable 
pattern to the movement of intensity of concern across types" (page 5); . t 

The CBAM researchers have identified seven different stages of concern 
about innovations under study. They confirmed, after many studies,- that • 
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"there is developmental movement through these stages" (Hall, George, and 

Rutherford, 1977, page 6). Their research ^hows that certain types of 

* 

teachers 1 concerns 'twill be more intense, then less intense, before arousal 

of other types wil,l occur" (page 6). Thus, these writers summarize: i 

Concerns about innovations appear to be developmental ia 
that earlier concerns must first' be resolved (lowered in 
intensity) before later concerns emerge (increase in inten- 
. sity). The research suggests that this developmental 
pattern holds for most process and product innovations 
(page 6). 

Hall, Geprge, and Rutherford stated that teachers 1 concerns are highly . 
complex and that their resolution is not simple. Possessingmore knowledge 
of, time with, or experiences with an innovation does not guarantee that the 
teachers 1 cqncerns will be reduced or resolved. They maintain' that "the 
process of t;Jie arousal and resolution of concerns is highly personal and 
requires time as well as timely intervention of bot^ cognitive and affective 

n&tures" (page 6). They report, though, that their research shows teachers 1 

« *■ " - 

concerns regarding an educational innovation increased in intensity in the 
later stages. This, suggests that given additional "time, successful 
experience, and the acquisition qf new knowledge and skill, Vteacljers' 
concerns will develop toward the stages related to impact concerns (page 6). 

Table 2 presents the seven identified and verified Stages of Concern 
about an innovation.. This table, provided by the CBAM project, presents the 
stage number, the name of each stage, and typical teacher expressions of 
concern about the innovation for each stage-. Relating' these seven stages 
back to Fuller's work, stages zero, one, and two relate to teachers' concerns 
about self; stage three relates to concerns of task ; and stages four, five, 



and six relate to concerns about impact of the innovation.' 
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Table 2 

Stages of Concern: Typical Expressions 
of Concern about the Innovation* 



Number 
of Stage 


Name of 
' Stage 


Typical Expressions of Concern Per Stage < 


0 


Awareness 


' I am not concerned about it (the inno- 
vation). ~ 


1 


Informational 0 


I would like to know more 'about it. 


2 


Personal 


How will using it affect me? 


*3 


Management 


I seem to be spending all my time in 
getting material ready. 


4 


Consequence 


How is my use affecting kids? 


5 


Collaboration 


I am concerned about relating what I am 
doing with what other instructors are 
doing. 


6 ' 


Ref oc using 


I have some ideas about something that . 
would work even better. 



♦Source: Procedures for Adopting Educational Innovations Program, 
Research and Development Center for Teacher Education, 
The University of Texas at Austin. 
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The instrument used to measure the seven previously-identified Stages of 
Concern was the Stages of Concern Questionnaire (SoCQ; Hall, George, and 
Rutherford, 1979). This instrument is a 35-item paper and pencil device 
♦developed over a period of two and ope half years at ttfe Research and 
Development Center for Teacher Education. Each of the 35 stimulus items 
solicits a response on a seven point Likert scale. Teachers 1 responses along 
the Likert scale indicate their degree of concern about the stimulus item. 
Test-retest study results produced $tB|e score correlations ranging from .65 
. to .86. , Alpha coefficient reliabilities ranged from .64 to .83 with six of 
the 1 coefficients being above .70. CBAM researchers conducted a series of 
validity studies which provided additional evidence that the SoCQ does, in 
fact, measure the hypothesized stages^of concern. 

Each teacher involved in the study completed the SoCQ instrument at the, 
conclusion of a taped interview (described next). The SoCQ can be scored by 
a computer program or manually and given the rather small number of teachers 
the manual method of scoring was chosen. Extra steps were added Ho help 
assure accurate scoring and interpretation. First, each teachers 1 SoCQ was 
scored manually three* separate times by the evaluator with a minimum of two 
weeks lapse between rescoring sessions. Second, SdC profiles were produced 
with codes rather than identifying names on them. Third, the evaluator 
sought and obtained analyses of the SoC profiles from two Qf the original 
developers of the SoCQ. These analyses were solicited in order to confirm 
the evaluator *s, interpretation of the SoCQ results. 

Teachers 1 Interviews . As part of the product evaluation, a series of 
structured interviews was held w>th the project teachers. The details of the 
focused nature of these interviews will be discussed later. The interviews 
ranged in time from 12 minutes up to 48 minutes with the average bejng 29 
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minutes long. Each cassette tape was reanalyzed a minimum of three times by 
one rater with a minimum of one week time delay between the listenings. The 
reason for the time delay was to allow recall of the tapes 1 content to become 
unfamiliar (in a cognitive sense) to the evaluator. In this way, each 
listening session produced "fresh? data which was compared to the previous 
listening for category coding agreements. 

In addition tb being used for product evaluation data collection, the 
tape recorded interviews with the project teachers were reanalyzed in order 
to collect process evaluation data. During the v focus$d interviews, it was 
discovered that the teachers rfiade numerous evaluative /comments about the 
Stal lings Classroom Management Staff Development Demonstration Project and 
their involvement in it. 'Although this was one focus>.area for the struc- 
tured portion of the interviews, what actually happened was that project 
teachers expanded on the topic, given the opportunity, and much more data 
became available for analysis. After the first reanalysis of the interviews 
was performed, the results were tabulated in the.form of data counts per a 
prel iminary set* of evaluative comment categories. Further, they were 
categorized by school and as positive or negative. The; prel iminary results 
were shared with the teacher trainer. The teacher trainer's recommendations 
were for specific new coding categories and a tally of the v number of teachers 
who provided data for each major category and its subcategories. The coding 
scheme was revised based on the teacher trainer's suggestions. Then, the 
interview tapes were reanalyzed at least two more times before the data 
collection was considered completed. The result was a table of eight major 
coding categories and a total of 36 subcategories (see Table 7, page 40). 
Through the interactions with the teacher trainer* the interview tape coding 
scheme for evaluative comments possesses high usability. The reanalysis of 
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the teachers' interview tapes produced a coding scheme which seemed to repre- 
sent the underlying categories. No other types of validity for the coding - 
scheme were obtained or reported. An instrument is said to be reliable when 
it measures what it purports to measure consistently. A degree of intrarater 
reliability was achieved in the reassessment pf the tape? several times by 
the single, rater with periods of delay \n between. One hundred percent 
agreement was reached for the second and third codings of the tapes. How- 
ever, no other formal reliability coefficients were computed. 

Product Evaluation Instruments 

As was the case with process evaluation instruments, two instruments? 
were utilized solely for the product evaluation while one instrument 
collected both product and ^ocess evaluation data. AJso, interview 
schedules were used for product evaluation data. Each*'pfoduct evaluation 
instrument shall be discussed in turn. 

Responsibility for Student Achievement Questionnaire . The Responsibility 
.ftfsf Student Achievement Questionnaire (RSAQ) iSsan instrument designed to 

f ' \; • 

I measure teachers' 1 perceptions t>f their responsibiji.ty for student, achieve- , " 

„. '* ' % , ,. .i ' • » > ' - 

ment in their classes (Stall inqs, Needels, and Stayrook, 1979). The RSAQ is 
B 30-item paper and pencildevice with a unique stjmulus and response format. 
[Each stimulus item is usually an uncompleted declarative; statement. For 
Example, item number two's stem is "When your class is having trouble under- 
standing something you have taught, H is usually blcause. . ." There are^btfo 
responses for each stem. To complete the example, the responses for item 
number two are:- "(a) because you did not explain it very clearly," and 
"(b) because your students are just slow in understanding difficult state- 
ments." The teacher is instructed to insert a weight, in percentage terms, 
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to each of the two choices according to his/her preferences. Again following 
the example, a teacher may respond 85% to response' M a n ^and 15% to response 
"b". In every case, both weights assigned to' the responses must add up to 
100%. / \ 

Scorinq the RSAO produces multiple scores. Positive or negative direc- 
tion is assigned to either response option from the teacher's perspective 
•(responsibility). Scoring the items and adding them results in two scores. 

The two scores relative to the teacher's responsibility for student achieved 

j i i 

ment are, first, the "teacher positive responsibility 11 and, second, its 
residual, the "teacher negative responsibility." for this evaluation, only 1 
the "teacher positive resppnsibility" was utilixed', although the lother score 
' was computed and is available for later analysis. The RSAQ has proved useful 
in past projects .(Stallings, Needels,. and Stayroo^, 1979). . Face^validity 
for this instrument's use in this particular evaluation was determined by 
1 the teacher trainer and the Superintendent of Schools: The J se two indivi- 
duals inspected each Item on the instrument and its^responses yi terms of 
their applicability .irr the .present situation. Both parties agreed that the 
items and responses on the RSAQ were applicable to .the local schools, theii. 
classrooms, teachers, students, and events/activities. No formal reliabili- 
ties were computed bu^Jthe hand scoring of the RSAQ was double-checked by 
another evaluator. This instrument was administered on a pre-pdsttest basis. 

Levels of .Use Ratings . The Levels of Use is the second dimension of the 
CBAM research described earlier. Whereas the Stages of Concern dimension "' 
measures the feelings, attitudes, and concerns about teachers' involvement 
in an innovation, the Levels of Use dimension describes what individuals 
actually do with an innovation. Eight operationally defined positions - 

< 

regarding teachers' levels of use have been identified from extensive 
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research (Loucks, Newlove, and Hall, 1975). These various levels represent 
a continuum from non-use, through mechanical use, refinement, to renewal . 
Further, seven critical decision points separate the levels and are part of 
the system. Table 3 presents the Levels of Use, their designations, and 
behavioral indices of each level. 

The eight Levels of Use, as depicted in Table 3, are subdivided further 
into seven categories of user descriptors producing a 56 cell matrix. These 
seven categories are knowledge, acquiring information, sharing, assessing, 
planning, status reporting, and performing. CBAM research has verified that - 
teachers at different Levels of Use of an innovation can demonstrate differ- 
ent behaviors within each cateqory. In summary, the Levels of Use, their, 

a 

categories, and the seven decision points have been assembled into a Levels, 
of Use chart (Hall, Loucks, Rutherford, and Newlove, 1975). 

The Levels of Use ratings research and development procedures were con- 
ducted over a two year, longitudinal study. Levels of Use interviews were 
t&pe recorded and subsequently rated by teams of two CBAM staff members. 
Resulting interrater reliabilities of the ratings ranged from .87 to .96 «, 
(Loucks, 1977). Interrater reliability on the overal l levels of n Use ratings 
was .96, with 73% agreement between the two raters (Loucks, 1977). Validity 
of the Levels of Use ratings was established using an ethnographic methodo- 
logy. A correlation coefficient between the interview ratings and a full 
day of observation of a sample of teachers at all levels of use was .98. 

Teacher behaviors related to Levels of Use of the demonstration project 
were measured by the Levels of Use Ratings. The data source for these Levels 
of Use Ratings was the focused interviews developed and tested by CBAM 
researchers (Loucks, Newlove, and Hall, 1975). The purpose of the focused 
interviews was to solicit sufficient information from teachers involved in 
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Table 3 

Levels of Use of the Innovation: Typical Behaviors* 

! . f 

1 i ■ ' 





Designation 


.Name of Level < 


Behavioral Indices Qf Level 




o. 


Non-Use' 

4 


No action is being taKen witn respect 
to the innovation. 




I, 


Orientation 


The user ^is seeking out information 
about the innovation. 




II 


Preparation 


r The user is preparing to use the inno- 
vation. 




III 

*- 


* Mechanical Use 


The user is using the innovation in a 
poorly coordinated manner and is making 
user-oriented changes. 


It".* ' v 


IV-A 


Routine 


The user is making few or no changes 
and has "an, established pattern of use. 




IV-B 


Refinement 


The user is making changes to increase 
^ outcomes. 




V 


Integration 


The user iV making*, del iberate efforts 
to coordinate With others in using 
the innovation. 




VI 


Renewal 

* 1 

\ 


The user is seeking more effective 
alternatives to the established use 
H of the innovation. \ 



♦Source: Procedures for Adopting Educational Innovations Program, 
Research and Development Center for Teacher Education, 
The University of Texas at -Austin 
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the innovation in order to place them at a Level of Use with respect to the . 
target innovation. Trained Levels of Use interviewers use a branching format 
based on the'decision points and they probe the categories for specific, *_ 
behavioral indicators of the interviewee's involvement with the innovation. 
The length of the focused interviews varies according to several factors and 
the interviewee may think the event is a casual conversation; however', 
because of the training -process and the fact that the Levels of Use chart is . 
memorized, the original target is never out of focus. 

The evaiuator was trained in th€"levels of Use interviewing procedures. 
This training included obtaining satisfactory interrater reliabilities with . 
the CBAM staff ratings of common interview tapes. The result of this 
training atad satisfactory interrater reliabilities was certification of this 
evaiuator as a "licensed" Levels of Use interviewer. Also, recall from 
previous sections that these tape recorded interviews were utilized for the 
collection of process evaluation data through reanalysis of the tapes. 

For this evaluation a demonstration project "user" was operationally 
defined as a teacher who had internalized a minimum of two variables on the • 
teacher prof ile chart >which should have been considered in lesson planning, 
and performed in subsequent classroom teaching. 

Secondary dbservation Instrument . Descriptions of the three part Secon-- 
dary Observation Instrument (SOI) were provided in previous chapters. The 
SOI is a low-inference category observation system developed over a period 
of ten years by Jane Stal lings- and Margaret Needels under a grant from NIE. 
Putnam County demonstration project coders were trained in the SOI obser- 
vation system by Stall ings and the teacher trainer. This coder training was 
' conducted onsite in Putnam County^ an intensive seven day period in January 
1981.- During the finalcoder training session, the trainees were evaluated. 

• 3 o » 



This 'evaluation of SOI coders included: (1) a written examination covering 
" ■ ■" . . ••' ■ - • , < * 

the coding categories, (2) interrater reliability checks, and (3) intrarater 

reliability checks. Each SOI coder passed the written examination at or 

above the pre-established, criterion score, Interrater reliabilities were 



based on the* coding of videotapes o^ classroom scenes and "oral) vignettes, 
Intrarater reliabil ity, checks were conducted using coder-partners observing 
a practice classroom situation. In both cases, the resulting reliabilities 



for the six coders ranged from .81 to\.91 



Teacher profiles resulting from thd> . Snapshot and the FMI components of 

\ ' 



the SOI, contained the recommendations of "More" or "Less" fpr each of the 45 
aggregated variables based on research findings; These pre-intervention vs, 
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post^tnrrervention comparisons of the "Morfes"< and "Lesses" produced the number 

of "correct implementations 11 of teaching behavior changes per each demonstra- 
tion' project teacher. These "correct impl^fnentations" resulting from the 
intensive observations served as one unit of data for assessing implementa- 

■ ' \ ■ ■ 

tion impact* \ > 

Administrators' Interviews , As a final measure of both process and 
product evaluation, selected project personnel * were interviewed. Although 

interview schedules were developed ahead of timle, the interviews were casual 

\ - 
and open-ended enough as to provide qual itative ^ata frpm the interviewees. 

Those demonstration project-related personnel included the two building 

. \ ■ - 

principals, the teacher trainer, and the Superintendent of Schools. A 

common core of auestions was asked of each interviewee in order to elicit ; 

related responses for later analysis. 
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Data Collection and Analysis 

Data collection procedures were rather straightforward. Some of the data 
collection and analysis ^procedures were discussed in the p|jgvious s£c|ions 
for each instrument. Basically, to avoid duplication of efforts and possible 
perceptions of "oyer involvement" of Outside personnel, the teacher trainer 
agreed to;administer several instruments in natural ly-occurring project 

events-such as the first and fifth small group sessions and the final meeting 

■\. ' . . - ■ • . it . 

of the total cjfoup. Additionally, the evaluator was introduced to the. 
building principals and the project teachers by the teacher trainer. Fol- , 
lowina the introductions, the evaluator conducted the Levels of Use inter- 
views and individually administered the SoCQ. The evaluator arranged the 
interviews with the principals, the teacher trainer, and the superintendent. 
As discussed earlier, the evaluator and the teacher trainer cooperated to 
make the interview tape reanalysis efforts more valuableTlo the total evalua- 
tion's usefulness in terms of providing information for decision-making. r . 

Given tile small number o^F project teachers (^clT),^most of the data 
analyses,, wer^ performed manually. Extra precautions wer*e implemented /because 
of these computation methods. Theise extra pjy^cedures included double and 
triple checks ©f the computations by the evaluator. Then, anpthfer experi- 
enced evaluatbrVchecked the ; evaluation procedures, the 1 ' d,ata manipulations, 
and the resultant dat$ displays. One part of the data.analy$is--those 
involving the correct implementations of teaching behaviors-.-was performed 

by a computer consultant using the Statistical Analysis System package and a 

4 ■ 

\ * 

custom written program. Actual computer analysis was performed on an IBM 
360-90 computer. 
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v > ' . CHAPTER I.V ; 

Process Evaluation Results 

The previous chapter described the evaluation design, the data collection 
instruments, and the data collection and analysis procedures of the evalua- 
tion of the Standings Classroom Management Staff Development * Demonstration 
Project in Putnam County Schools.. Process evaluation was defined as the 
assessments of the processes and activities experienced by project partici- 
pants and not the objectives, > renorts, schedules, etc. topically included jn 
a traditional evaluation of project processes. This chapter will present 
the process evaluation results via data displays and narratives grouped by 
the data collection instruments. 

» 

Reaction Survey , 

Results of th^administration of the Reaction Survey are- presented in 
Table 4. Data displayed in Table 4 shows that the demonstration project 
teacher? 1 feelinqs and/or reactions changed as the project activities 

unfolded. It shogjd be noted that one teacher did not complete the reaction 

*** ^ 

' survey.' On the scale of "+3" to. M -3Vit can be observed that the teachers' 
initial feelings were collectively at the +0.5 position. As the project 
activities unfolded* teachers 1 feelinqs and/or reactions moved up and down ■ 
slightly until the second half of the project where their feelings/reactions 
moved up antf reached the highest point value of +l.fi.at the end. A paired 
t-test of the difference between the event number one scores and the event 
numher twelve scores produced nonsignificant results. The lowest mean score 
of the qroup was for event' number five, After the First Teacher Training 
Session. This is tf\e session where the computer-qeneratpd teacher behavior 
profile charts were disseminated. These teacher profile charts contained 



Table 4 



Descriptive Statistics on Teachers 1 Feelings and/or 
. " Reactions as the Project Unfolded (N=10) 



Number Description of Event Mean 3 Standard 

, * • Deviation 



1 

1 


Del Or c Lilc dWdrcilcbb bebb lOil % 




1 97 


0 

c 


hi Lcr trie awarciicbb btjbbion 




1 • 1 D 


3 


Before .the first set of observations 


-0.1 


i 

1.20 


4 


After the first set of observations 


+0.3 


1.25 


5 


After the first teacher training session 


-0.2 


1.40 


6 


After the third teacher training session 


+0.4 


1.51 


7 


After the final teacher training 1 session 


+0.7 


1.42 


8 


Before the second set of observations 


+0.9 


1.45 


9 


After the second set of observations 


+0.9 


1.52 


10 


* < 

Before receiving final profile 


+0.7 


1.83 


n 


After receiving final profile 


+1.4 


1.71 

V 


12 


Today: 6/8/81 


+1.6 


1.58 



a Scoring was from +3 to -3. 
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* 

the recommendations of "More" or "Less" for changes in teaching behaviors in' 

the classrooms. • . 

/ 

Fiqwre$_2 and 3 present the same data from the Reaction Survey but 

: • l . . 

displayed much differently. Figure 2 is a gra^h depicting the teachers 1 
feelinqs and/or reactions &s the project unfolded. ;Each teacher's response 
position on the scale for each evertl is denoted by a solid circle. Further, 
the qroup's mean score is plot-ted dn the chart with a line. Thus,' the move- 
ment of the group's feel ings/ifeactions, as reflected by their prose responses, 
can be seen easily. The' lowest point of their feelings/reactions is shown in 
the chart as is the slow rise to a final value of a little past the midpoint 
between +1 and +2. Figure 3, on-the otheV* hand, is a chart depicting each 
teacher's feelings and/or reactions as the project activities unfolded. .Here 
each individual teacher's responses to the stimulus items are depicted by a 
different type of line. Thus, each individual teacher's self-reported 
feelings and/or reactions to the project as -it- unfolded are displayed. This 
type of display protects the anonymity of the teachers, yet still present^ 

V 1 r 

an accurate display of individual responses. 
Stages of Concern 

Results^ of the end-of -project administration of the SoCQ are presented 
three different ways from the most global analysis of- data to an item 
display. Figure 4 displays three Stages of Concerh profiles: for each of 
the two project schools and both schools combined. Displaying the SoC data 
by the two schools was completed for the superintendent's convenience. The 
relative intensity of the teachers' concerns are denoted in percentiles on 
the vertical axis. The seven Stages of Concern are denoted on the horizontal 
ax i s . 

4,i 
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•aclng Is not proportional 
i tine Intervals 



Figure 2 • 

Graph Depicting Teachers 1 Feelings and/or Reactions 
as the' Project Activities Unfolded 



KEY: • = Individual 
wmm = Mean 
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Chart Depicting Each Teachers 1 Feelings and/or 
Reactions as the Project Activities Unfolded 
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Analysis of the three Stages of Concern profiles are warranted. For both 

1 r v 

schools* teachers combined, the most intense concern was Informational at the 

w 

88th percentile.' This indicates an interest in more information about the 
project." The concern is not so much for details as- it is for general infor- 
mation about the innovation. The second most intense concern was Personal at 
the 84th, percentile. This indicates high arousal of ego-centric, personal 
feelings and attitudes about the project . Concern here is about the'statljs, 
reward, and effects of the demonstration project on the teacher. Not.much 
lower (80th percentile) was the third most intense concern of Collaboration. 
This indicates interest regarding collaborating with other teachers relative- 
to the project. Interestingly, the least intense stage was the Management 
concerns *at School B. •• * . 

Table 5 displays the statistics from the administration of the SoCQ. 
Shown are means, standard deviations, and percentiles for each school and 
both schools combined for each stage and the total SoCO score. Also, the 
t-values and significance levels for each SoCQ stage and the total SoCQ 
score across schools are presented. Data in Table 5, show-only stage number 
two— Personal— produced a significant t-value.- Inspecting the percentile 
columns shows teachers in School B had the most intense concerns at Stage 
Two. Recall that Stage Two concerns deal with the self in relation to the 
innovation. Here the individual teacher is uncertain about the demands of 
the innovation, his or her adequacy to meet those demands, and his or her 
role in the innovation. This includes analysis of his or her role in rela- 
tion to the reward structure of the organization, decision-making, and con- 
sideration of potential conflicts with existing structures or personal 
commitment (Hall, George, and Rutherford, 1979). 

4j 



\ Table 5 

Statistics from the Stages" of Concern Questionnaire 



• 

Number 
of Stage 

» 


Name >of 
' Stage 


Both Schools 


/ M-1 1 \ 

IN-l l ; 


School A (N=7) 


School B (N=4) 


t-yalue 
(across 


Sia. 
Level 


Mean 


SD 


%L* 


Mean 


SD , 


%L* 


Mean " 


SD , 


%L* 


o 


Awareness 


9.18 


5.67 


77 


11 .43 


5.71 


84 


5.25 


3.10 


53 


2.10 


NS ' 


. 1 


Informational 


23.09 


3.99 


OA 


23.71 


4.99 


88 


22.00 


0.82 


80 


0.67 


N$ 


2 


Personal ( 


23.73 


7.94 


83 


20.00 


6.83 


72 


30.25 


5 .25 


94 


2.59 


,05** 


3 " 


Management 


14.91 


10.02 


-x56 


19.14 


7.82^- 


73 " 


7.50 


9.88 


27 


2.18 


NS 


4 


Conseauence 


29.91 


5.13 


76 


28.57 


5.88 


71 


32.35 


2.63 - 


86 


1.17 


NS 


5 


Collaboration 


28.00 


6.62 


'80 


26.86 


8.11 


76 


30.00 


2.45 


88 


0.28 


NS 


6',. . 


Ref ocusinq 


20.45 


7.35 


65 


. 22.71 


7.20 


77 


16.50 


6.61 


47 


1.42 


NS 




TOTAL. SCORE • 


149.27 

• 


18.57 


86 


152.42 


19.64 


89 


143.85 


17.71 


83 


0.73 


NS 



^Denotes percentile. , 
**t.05(9) - 2.26 ■ 



"V ' < 38 

\ * 

» v . 
»" " 

•" • ' 1 ' 

Table 6 presents the comparisons "of SoC'0, Stage Two (Personal) items 
across the two schools. The five items making up- Stage Two are presented 
together with their means.,- standard deviations, t-values, and significance 
levels. Data in Table 6 shows that SoCQ items 7 and 33 produced significant 
t-values. In both cases, the School B teachfrs 1 concerns were significantly 
higher than School A teachers 1 concerns. School B teachers expressed signi- 
ficantly higher concern score means on t^ese two SoCQ items: ■ » • 

7. I would like to know the effect of reorganization 
on my professional status. 

"J ■ ; 

| ^ 33. I would like to know hbw my rolejwill change when 
! I am using the innovation. 

Both jpf these Stage -Two items are related in that they deal with the position 

of the individual in the organization. These items mention either profes- 

I ~ | j 

sion|n stttus or role change. Inspection of ITable 6 shows the faculty of 

School B to be significantly more concerned about the effect of the demon-' 

i j 
.stration project on their professional status and role with4n the organization, 



TeacjhersV Evaluative Comments \ , 

( Table 7 displays the results of the coding V The teachers 1 . interview 
tap^s for assessment and evaluation comments regarding their participation 
inithe demonstration project. Data are presented in eight major categories 
and 36 subcategories derived from several listenings and discussions of 
usefulness with the teacher trainer. Teachers evaluative comments are 
presented in Table 7 by the school, whether they were primarily positive or 
negative, the number of teachers indicating, and the number of separate 
pentions per each subcategory response. 

/. A. total of 685 evaluative comments were gleaned from the eleven teachers* 

/■ 

interview tapes. School A faculty provided a total of 562 of those comments 

) 
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Table 6 " ' 

Comparison of SoC Questionnaire, Stage 
Number 2 Items Across Schools 



,39 



\ 



CY 



SoC \ 

Item SoC Statement* 
No. 



School A (N=7) School B lN=4) 
Mean SD Mean SD 



t- Sig. 
Value Level 



7. 



13. 



17. 



28. 



33. 



"I would like to know 
the effect of reorgan- 
ization on my profes- 
sional status." 

"I would like to know 
who will make deci- 
sions *in. the new 
system." 

"I would like to know 
how my teaching or 
administration is 
supposed to change." 



2.. 43 



3.71 



3.86 



"I would like to have 
more information on 
time and energy commit- 
ments reciuired by this - 
innovation." 5.43 

"I would 1 ike to know 

how my role will 

change when I am using 

the innovation." 4.57 



1.99 6v25 0.50 3.71 .01** 



2.69 4.25 2.75 0.51 - NS 



2.61 6.50 l'.OO 1.32 NS 



2.30 3.25 2.63 -1.45 NS 



1.81 6.75 0.50 2..32 .05** 



♦CoDvriqht, 1974, Procedures for Adopting Educational Innovations/CBAM 
Project, Research and Development Center for Teacher Educatnon, The 
University of Texas at Austin. 



**t 05 (9) = 2.26 
t;oi(9) = 3.25 
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Table 7 



Coding of Teachers' Evaluative Items from Interview of Tapes (N=ll). 









School A 


(N=7) 






School 


B- (N=4) 




Teachers 1 Assessment Information: Categories. and . . 


Positive 


Negative 


Positive 


Negative 


Subcategories 


No. of 


No. of 


No. ofc 


No. of 


No. of 


No. of 


No.' of 


[ No. of 






Teachers 


Separate ' 


Teachers 


Separate 


Teachers . 


Separate 


Teachers 


Separate 






Indicating 


Ment ions 


lndicat ing 


Ment ions 


Indicating 


. Mentions 


Indicating 


Mentions 




f 




(•) a 










\ 


■ (\) u 


A. Workshops ' 






■ 












1. 


Workshop leader as a person 


6 


10 (2.7) 






J* 




\ 




2. 


Workshop leader *s content knowledge 


i ■ - 


i QUA} 






c 






* 


3. 


Workshop, leader 1 s strategies 


' 7 


10 (2.7) 


3 


18 (9.4) 


3 


6 (5.0) 






4. 


Atmosphere in workshop 


2 


3 (0. 8) 














S. 


Content of workshops, including activities 






• 














related to variables 




-m t o 1 1 
(8 . 1 J 




1 (0.5) 


4 


20 [I6^j 






\6. 


Handouts and resources: 






J ' 








> 






(a) Immediate usefulness/appl Icabil J ty 




*5* (6.7) 


y u 


19 (9.9) 


3 


1? (10.1) 








(b) Future usefulness/applicability 


5 


29 (7.8) 


i 


1 (0.5) 


4 


9 (7.6) 






7. 


Teacher Behavior Profile Charts: 




















(a) Distribution at first workshop 






3 


10 (5.2) 












(b) Interpretation of 


4 


22 (5.9) 


5 


17 (8.9) 


2 


6 (5.0)/ 








(c) Usefulness of 


6 


24 (6.5) 


3 


5 (2.6) 


2 


5 (4.2) 




t 




(d) Attitudes toward • 




Ml (2.7) 


•T 

/ „ 


to { o . 1 ) 


I • 


*J ft T 1 






8. 


Future implementation of processes 


. 2 


7 (1.9) 


1 


1 (0.5) 












' SUBTOTAL 




179 (48.2) 




88 (46. 1 ) 




63' (52. 9) 




0 (0.0) 


B. Timing 


















1. 


Weekly scheduling of workshops < 


* ■ 




3 


21 (11.0) 








3 (75.0) 


* 2. 


Project start date in relation to. school year. 


2 


3 (0.8) 


6 


17 (8.9) 






2 




' ■ ' ' SUBTOTAL 




3 (0.8) 




38 (19.9) 




o (0.6) 


- 


3 frs.ny 


C. Coders and Coding 
















) 


1. 


Observer/coder selection process 




12 (3.2) 














2. 


Presence of coders in classroom^ 


3 


9'(2.4) rt 


1 


1 (0.5) 










3. 


Coding/ observation process 


5 ' . 


19 (5.1) 


4 


20 (10.5) 












SUBTOTAL 




40 (10.7) 




21 (IkO) 




0 (0.0) 




0 (0.0) ^ 


D. Sharing Information, etc. ^ 


















" ' . 1. 


Opportunity to meet as a subject area faculty 


6 


16 (4.3) 






1 


3 (2.5) 






2. 


Sharing during workshops 


5 


7 (1.9) 


< i 


4 (2.1) 


2 


5 (4.2) 






3. 


Sharing informally at school 


5 


16 (4.3) 




2 (1.0) 


4 


18 (15.1) 






4. 


Desire for common lunch or prep. 


ft 


8 (2.2) 






1 


1 (0.8) 






5. 


Building, muxiiai support sVs^em^ • ^ r - - 




.2l>-(S..t)*. 




3 (K6) 


2 


5 (4.2) 








■ * SUBTOTAL 




68 (18. 4), 




.9 (4.7) 




32 (26.8) 


* 


0(0.0) 
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Table 7 (continued) 







School A 


(N=7) 




School B (NM) 


Teachers' Assessment Information: Categoric^ and 
Subcategories 


Positive 


Negative 


Positive 


Negs 


itive 


No. of 

Teachers 

Indicating 


No. of 
S*jrarat e 
Ment ions 

m a 


No. of 

Teachers 

Indicating 


No. of 
Separate 
Ment ions 
C) a 


No. of * 
Teachers 
Indicating 


No. of 
Separate 

Mentions 


No. of 

Teachers 

Indicating 


No. of 
Separate 
Ment ions 
(*) a ' 


it 

E. Financial Aspect 

{. Amount of money spent 

2. Source of money spent 

3. Returns on money spent 

SUBTOTAL 


1 


1 (0.3) 
1 (0.3) 


3 
3 
3 


4 (2.1) 
6 (3.1)- 
6 (3.1) 


f 

\ 










16 (8.3) 




0 (0) 




0 (0.0) 


F. Information Seeking 

1. About the overall project 

2. About the coding system 

3. About the teacher profile charts 

4. About the project-related references 

* SUBTOTAL 


3 
1 
1 
1 


5 (1.3) 
1 (0.3) 

1 (0.3) 

2 (0.5) 


3 


4 (2.1) 


% 

3 
2 


2 (1.7) 
2 CI. 7) 




* 




• 9 (2.4) 




4 (2.1) 




4 (3.4) 




u i u . u J 


G. Miscellaneous 

1. Principal's support 

2. Students 1 reactions to new behaviors 
\. Parents' comments to teachers 

4. Hiring of substitute for classes 

5. Traveling to workshop site 

6. Desire for supplementary teaching materials 

7. Volunteer teachers in program 

SUBTOTAL 


2 
6 

, 2 

2 
1 


2 (0.5) 
24 (6.5) 

2 (0.5) 

13 (3.5) 
1 (0.3) 


* 

1 \ 


1 (0.5) 


4 

2 


» 

6 (5.0) 

3 (2.5) 

i 


1 


1 (25.0) 


42 (11.3} 




1 (0.5) 




9 (7.5) 




1 (25.0) 


H. Overall Impressions/Reactions 

1 r SUBTOTAL 


6 


29 (7. 4 8) k 
29 (7.8) 


3 


14 (7.3) 
14 (7.3) 


3 


11 (9.2) 
11 (9.2) 


\: • 


0 (0.0) 

i 


TOTAL 

(% of total Pos. 6 Neg. statements) 

h 




371 (99. 9^ 
(66.0) 




191 (99.9^ 
(34.0) 


< i 


119(99. B) b 
(96.7) 


J 


4 (100) 

(3.3) 

4 ■ 



^Reported a percentages within this column only. 
b Does not equal 100 due to rounding. 
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with 371 (66%) coded as positive and 191 (34%) coded as negative comments.. 
In contrast, School B faculty provided a total of 123 evaluative comments 
with 119 (97%) coded as positive and a mere 4 (3%) coded as negative. The 
-chi-square value for the total comments was significaTiFlit^the .001 level 
[x 2 (1) = 45.31]. There was a significant relationship between school 
faculty and the number of positive or negative comments provided in the 
taped interviews. School A made significantly more negative comments. 

The classroom management staff development teacher training workshops was 
the coded category for many of the taped evaluative comments. For School A > 
teachers, 179 (48%) of their positive comments were coded into this category 
and 88 (48%) of their negative comments were coded into this category. For 
School B teachers, 63 (53%) of their "positive comments were coded into this 
category and none of their , total of four 'negative comments was coded into 
the category. 

Within the workshops category, the subcategories present interesting 
data. The workshop leader as a person and her content knowledge received all 
positive comments'; however, three teachers in School A made 18 negative 
comments about the', -workshop leader's strategies.' Recall ttiese strategies 
- were mandated by the model itself and were not part of the leader's responsi- 
bility to change. The content of the workshops received many more positive 
comments than negative ones. The workshops '. handouts and resources received 
mixed reactions. Six of the School A teachers made 25 positive comments 
about their immediate usefulness/applicability but six teachers from the 
same school made 19 negative comments. Regarding future usefulness/applica- 
bility, five School A teachers made 29 positive comments and only a single - 
teacher made a single negative comment. 

- - 5u 
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Within the- workshop, the teacher behavior profile charts elicited enough 
comments to warrant further subdivisions. Ten negative comments- were coded * 
from three teachers for the subcategory of the teacher behavior profile ch^art 
being distributed at the first workshop. Four School A teachers made 22 (6%) 
positive comments regarding the interpretation of the profile charts, five 
School A teachers made 17(9%) negative comments, and two School B teachers 
made 6 (5%) positive comments about the interpretation of the teacher behav- 
ior profile charts. The usefulness of the profile charts was the subject of 
24 (7%) of School A teachers' positive comments from six teachers and only 5 
(3%) of their negative comments. In terms of teachers' attitudes toward 
their profile charts, three School A teachers made, lDU3X^positive comments 
but seven School A teachers made 16 (8%) negative comments. - 

Timing of^tfie demonstration project was the second category of coded 
comments. A total of 21 (11%) of School A teachers' comment/ were coded 
* \ as negative regarding the weekly scheduling of the teacher graining work- 

shops. The project start 'date in relation to the -schbol year (it started 
l^e as reported in the. narrat ive report) generated many negative comments 
and few positive comments: A total of six different School A teachers made 
17 (9%) negative comments while two of the four School B teachers made three 
(75%) negative comments. Only two teachers (School A) made a total of three 
(1%) posit ive; comments. 

The" demonstration project's coders and coding process was the third cate- 
gory of teachers' evaluative comments. School A teachers supplied all the 
assessment/evaluation comments in this category; School B teachers made none. 
For the whole category just as many positive as negative comments in percen- 
tage terms were produced (11%). The teachers had all positive comments about 
the observer/coder selection process. Presence of the coders in the 

ERIC 5 j- 
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classrooms generated nine (2%) positive comments' and only a single negative 
comment. The coding/observing process, produced 19 (5%) positive comments 
and 20 (11%) negative assessment/evaluative comments. Most of these 20 
negative comments reflected a lack of knowledge about the coding process. 

Sharing information re]ated to the demonstration project and its compo- 
nents was the fourth category of evaluative comments. The opportunity to 
meet as a subject area faculty received all positive comments (N=16) from 
School A teachers. Sharing during the wqrkshops received seven (2%) positive 

and four (2%) negative comments from School A teachers. Sharing informally 

f 

at school produced 16 (4%) positive and only 2 (1%) negative Comments from 
School A teachers. Contrastingly, all four School ,B teachers made 18 (15%) 
positive comments related to scaring informally at school. The desire to 
have a common lunch or preparation period was voiced eight (2%) times by six 
of the seven School A teachers. The building of a mutual support system 
generated 21 (6%) positive comments from, six School A teachers and just thpee 
(2%) negative commerfts while half the School B teachers provided five (4%) 
negative comments. Taken as a' whole, the sharing information category 
produced 21 (27%) of the School B positive comments. 

The financial'aspect of the demonstration project produced very dif/erent 
results from the two schools' teachers. School B teachers did not make any 
positive or negative comments about this topic. A few School A teachers 
made numerous negative comments about the financial aspect including the 
amount of money spent (2%), -the source of the money spent (3%), .and the , . 
returns on the money spent (3%). In each case, all the negative comments 
were made by three of the seven School A teachers. 
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Information seeking'was the fifth category Qf' taped evaluative comments 
about the/j*roject. Overall this category produced very few comments from 
both s<fhool faculties. • 

/ The miscellaneous category contained a "mixed bag" of subcategories. 
Most of the comments were positive and from the teachers in School A. Inter- 
estingly, six of the seven School A teachers commented positively (N=24, 6%) ■ 
on students 1 reactions to the teachers' new behaviors. All four School B v 
teaches made positive comments (N=6, 5%) on students 1 reactions to the ,* 
teachers' new behaviors. 

Overall impressions and/or reactions was the last category of interview- 
produced assessment or evaluative comments. Six of the School A faculty made 
29 (8%) positive comments and three made 14 (7%) negative comments. Three of 
the four School B faculty made 11 (9%) positive comments and no neg-ative 
statements. 
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' CHAPTER V 
Product Evaluation Results 

* The previous chapter presented the process evaluation results. This 
chapter will present the. demonstration project's product evaluation results. • 
Data displays and narrative copy will be used to present the product evalua- 
tion results. 

Teachers' Responsibility for Student Achievement 

• Table 8. displays the pretest ANOVA and the posttest ANCOVA tables from 
the analyses of the Responsibility for Student Achievement Questionnaire 
(RSAQ). The pretest RSAQ results revealed a significant F-ratio (F (1, 8) 
8.15, p< .05) across the' two schools' faculties. This necessitated the use 
of the Analysis of Covariance (ANCOW) statistical technique with the pretest 
RSAQ used as the* covar,iate. With the\ pretest RSAQ score as the covariate, 
the posttest RASQ scores still produce^! a, significant F-ratio (F (1, 7) = 

i 

f 

5.60, p < .05). 

Table 9 presents the pretest, posttist, and adjusted posttest data from 
the teachers' RSAQ administration. Data! in Table 9 show that the mean RSAQ 
score, for .both School A and School «B" incrleased; 4 Ttiis difference in adjusted- - ... 
posttest RSAQ mean scores, based , on pretext scores, was ^significant at the 
.05 level . . \ ' . . 

Levels of Use Ratings 

Results of the Levels of Use about the innovation ratings from the focused 
interviews conducted at the conclusion of the project are presented in Table 
10. Here it is shown that overall there were: two teachers at the Non-Use 
level, one teacher at the Orientatation level, three teachers at the Mechanical 
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, Table 8 

ANOVA arid ANCOVA Tables for the Teachers" Responsibility 
for Student Achievement Instrument Across Schools 



Source 


Sum of 
Sauares 


,df 


Mean 
Squares 


F-Ratio 


Sig. 
Level 






Pretest 


ANOVA 






Treatment 


558.2 


< 

1 


558.2 


8.15 

•M 


.05 a 


Error 


548.2 


' 8 


68.5 


% 




Total 


1,106.0 


9 












Posttest 


ANCOVA 






.Total 


1 358.0 ' 


\s ■ 


28. 4~^ 






Error 


r 199.0 


7 ' 








Treatment 

u 


159.0 , 


1 / 

M r 


159.0 


. 5.60 


.05^ 



a F.05O.8) 
b F. 05 (l. 7 ) 



5.32 
5.59 



\ 
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Table 9 



Pretest, Posttest, and Adjusted Pbsttest Data from the Teachers',, 
Responsibility for Student Achievement Instrument Across Schools 



Statistic 


" 1 

Pretest 


Posttest 


Adjusted rosttest 


School A 


School B 


School A 


School B 


School A School B 




6 


4 


6 


4 




I 


52.5 


67.7 


52.2 


74.3 


57.1 69.4 


S,D 


8.7 


8.0 


5.9 


9.1 




Range 


42.0-63. 


7 58.7-68.6 


43.7-60.3 


62.7-84.0 1 




F -Ratio . 




8.15 




NA 


- 5.60 


, Sig. Level 




.05 




NA 


.05 
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v Table 10 
Levels of Use <sf pf-*the Innovation Ratings (N=ll) 



Desig- 


Name of Level 

S W 


« Both Schools 
. (N=ll) , 


School A 
CN-7) 


School B 
(N=4) 


nation 


No. 




No. 
— i- 




No. 


% 


0 


Non-Use 


2 


18.2 




28.6 


0 


0.0 


I 


Orientation 


1 1 


9-1 


1 


14.3 


0 


'O.O 


II 


Preparation 


0 


0.0 ' 


0 


0.0/" 

y 


0 


0..0 


III 


Mechanical Use 


3 


27.3 


2 , 


28.6 


1 


25.0 


IV-A 


Routine 


{ 5 


45.5 


2 


28.6 


3 


75.0 


IV-B 


Refinement 


^ ■ 0 


0.0 


0 


0.0 , 


0 


0.0 


V 


Integration 




0.0 


0 


0.0 


0 


0.0 


VI 


Renewal 


0 


0.0 


0 


0.0 


0 


0.0' 



a Does not total TOO due to rounding. 

i 

NOTE- The dotted line separates innovation users from non-users: above the 
dotted lines are non-user categories, below the dotted line are users. 
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Use level, and five teache^sat the Routine level. Levels of Use ratings 
across the two schools produced differentiated results. There were no non- 
users from School B:\ the three non-users were all from School A. Also, 
School B had three teachers' at the Routine level (IV) while School A had 
two. When the non-user categories and the user categories were collapsed 
then compared across schools School A had more non-users than did School B. 

Changes in Teachers' Classroom Be^viors 

Demonstration project teachers! classroom behaviors were' observed using 
the Secondary Observation Instrument (SOI). XlasTrtgps^efe^Dbserved three 
class periods before the teacher training workshops and three class periods 
after the teacher training workshops. These teacher training workshops were 
the interventions attempting to help teacherls change some of their specific 
classroom behaviors in- ways suggested by research results. Technically, 
these observations should be labeled "pre-intervention observation" and 
"post-intervention observation." However, for the reader's convenience, they 
will be labeled with the shortened terms of "pr|-observation" and "post- 
observation." A total of 19,855 teacher-focused FMI interactions were coded 
from the project teachers' pre- and post-observations. These classroom ^ 
"observation verbal interactions averaged 903 per each teacher's pre- and 
post-observation sets. Ih'is ' averages' out to 301 verbal interactions, exactly 
one more than expected by the SOI system. The^umber of coded interactions 
per three day observation sets (pre- and post-observations) ranged from a 
low of 699 to a high of 1,132 specific interactions. Additionally, 330~5ttf— 
Snapshots were completed which detailed classroom activities by percent of 
time involvement. , 
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A total of over 140 specific variables are produced by the SOI'^FMI com- 
ponent. More than 15 Snapshot variables are coded. After data analyses, the 
FMI verbal interactions are aggregated into 30-35 variables for the teacher 
behavior profile chart.* Also, the Snapshot yields 14 variables for its own 
teacher profile chart-. These two SOI-produced teacher profile charts are the 
basis for the content of the teacher-training workshops. 

The demonstration project utilized 45 FMI and Snapshot variables as 
targets for analysis, discussion, and change. Stal lings (1980). grouped the ✓ 
45 variables into three major classification types: (a) interactive instruc- 
tion variables (N=28), '(b) non-interactive instruction variables (N=8), and 
(c) off-task variables (N=9). As a result of working with the 45 variables 
"in the Putnam County teacher training workshops in the demonstration project, 
the teacher trainer. sought to organize the' variables into a complimentary 
grouping scheme, /Through several meetings, ,the teacher trainer and the 
evaluator developed another grouping scheme consisting of three categories. 
There was 100 percent agreement between the. two persons regarding this group- 
ing. The three groups were: (a) instruction variables (N=20), (b), classroom 
management, variables (N=17), and (c) feedback and discipline variables (N"8). 
The division of the 45 SOI variables into the two major grouping schemes 
appears in Table 11. These. two major grouping schemes and their subdivisions 
helped to organize the data analyses regarding the impact of the project on 
changing teachers' classroom behaviors. 

The unit Qf measure for studying the changes in teacher '&, cl assroem 
behaviors-reauires an explanation. Recall that the FMI and Snapshot pre- and 
post-observation's yielded teacher behavior profile charts displaying results 
of' sets of classroom observations. These two profile charts— one for the 31 
FMI variables and one for .the 14 Snapshot variables— display the individual's 



Table 11 

Stal lings Staff Development Program Teaching Variables 
(N=45) Listed by Two Major Grouping Schemes 



* 

Variables Grouped by Three Classification Types 


Variables Grouped by Three Categories | 


Interactive Instruction Variables (N-28) 


Instruction Variables (N-20) \ . 


F5 Adul % t to Individual Student 

F6 Adult jto Groups 

F7 Adult to Class 

F8 Student Direct Question/Reading 

F17 Adult Direct Question/Reading 

F25 Student Response/Reading v 

F43 Student Reading Aloud 

F45 Adult Instructing Group 

F48 Adult Instructing Groups/Reading 

F49 Adult Instructing, Everyone/Reading 

F61 Adult Acknowledgement /Reading 

F71 Adult Praise/Support 

F94 All Interactions/Reading 

F95 All Interactions/Task 

F99 All Interactions/Positive 

trinfi CtnHDnf Pnrnmwits Assignments 

F120 All Interactions/Class Assignment 

v\*\f* Arfnl r'/Di f f *»rent Student Starts Interaction 

F137 Different Student/Adult ?tarts Interaction 

F138 All Instruction 

F139 All Supportive Corrections 

Fl,4l Making Assignments 

54 Tota*l Reading Aloud 

55 Total Making Assignments » ■ ■ 

56 * Total Instruction ; 

57 Total Discussion 

58 'Total Practice Drill ^ 
Sll Total non-Math or Reading Instruction 


F8 Student Direct Question/Reading 

F17 Adult Direct Question/Reading 

F25 Student Response/Reading .* 

F43 Student Reading Aloud 

F45 Adult Instructing Reading 

F48 Adult Instructing Groups /Reading 

F49 Adult Instructing Everydtie/Reading ' 1 

pen kU.Mk^nA Tnc frnrf i np 9 

rju nacnxne AiioLiUULAMjj i 
F94 All Interactions/Reading 

pqC All Tnf •rarti nnR / Tlt;k 1 

F108 Student Comments Assignment 
F138 All Instruction 

53 Total Silent Reading 

54 Total Reading Aloud 

56 , Total Instruction 

57 Total Discussion 

58 * Total Practice Drill 

59 Total Written Assignments 

510 Total Test Taking 

511 Total non-Math or Reading Instruction 


Classroom Management Variables (N-17) j 


F5 Adult to Individual ' Student - 
F6 Adult to Groups 
F7 Adult to Class 
F56' All Social Comments 
F91 AIL Adult Movement 
- F120 All Interactions/Clas's Assignment 
F122 Adult Manage ( Class/J)!o Student 
F135 Adult with Outside Intruder 
F136 Adult/Different Student Starts Interaction 
F137 Different Student7Adult Starts Interaction 
F141 Making Assignments 
•F142 All Intrusions 

S2 Teacher Class Manage/No Students 

S5 Total Making Assignments * ♦ 

512 Total Social 1 Intervention 

513 Total Student Uninvolved 
S15 Total Classroom Management 


Hon- Interactive Instruction variaDies o; 


F50 Machine .Instruction 

F91 All Adult Movement 

F122 Adult Manage Class/No Student 

52 Teacher C.lass Manage/No Students 

53 Total Silent Reading 

S9 Total Written Assignments 

SIO Total Test Taking 

S15 Total Classroom Management 


• Off-Task Variables (N-9> 


Feedback and Discipline Variables (N-8) 


F56 All Social Comments 

F96 All Interactions /Behavior 

F102 All Interactions/Negative 

F135 Adult with Outside Intruder 

F140 All Correctives 

F142 All Intrusions # 

512 Total Social Interaction 

513 Total Student Untavolved 

514 Total Discipline 


F61 - Adult Acknowledgement /Reading* 
F71 AduU Praise/Support 
F96 All Interactions/Behavior 
F99 All Interactions/Positive 
' F102-A11 Interactions/Negative 
F139 All Supportive Corrections 
F140 All Correctives 
S14 Total Discipline 



Note: F - FMI variables " 

S - Snapshot variables 
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position relative to the norms for teachers 1 behavior from the baseline 
research project. Based on classroom teaching behaviors known to impact on 
student achievement outcomes at three levels of gain,s, three types Of recom- 
mendatiohs were made directly on the prof ile.charts. These recommendations 
werg for "More," j"Le$s," or "OK^ regarding changing the teaching behaviors 
during the time span before the post-observations. There was a column for 
these recommendations on each profile chart. For the purposes of this 
analysis, the "OK" recommendations on the pre-intervention profiles were 

counted as "correct implementations" of teaching behaviors. Similarly, the 

< * .. 

r ... 

post-observation FMI and Snapshot profiles contained the same recommendations 

.... * '** f 

of "More," "Less," or "OK,. 1 ' Here the Correct implementations" on the pro- 
file charts were denoted by asterisks. Thus, there was a possibility of 45 
"correct implementations" of teaching behaviors at both pre- and post-obser- 
vation times. 

Table 12 presents the pre- and post-observations of the correct implemen- 
tations of all 45 teaching behaviors across both high school? . Here it is 
shown that the mean number of correct implementations of pre-observations ^ 
teaching behaviors for School-teachers was 22.00, while the mean number of 
correct implementations of teaching behaviors for School B teachers was 
21.25." There was no significant difference between the mean number of 
Correct implementations of teaching behaviors across the two schools 1 
teachers. Table 12 also shows that the mean number of correct implementa- 
tions of post-observation teaching behaviors for School A was 30.71, while 
the mean number of correct implementations of teaching behaviors for School 
B was 31.00. There was no significant difference between the mean number of 
correct implementations of teaching behaviors across the two schools 1 
teachers at post-observation. 



Table 12 



Statistics from the Pre- and Post-Observations of Correct 
Implementations, of all Teaching Behaviors Across Schools 



Timing 


School A 
(N=7) 


. School B 
(N=4) 


t- 


Sig. 


Mean SO 


Mean SD 


. Value 


Level 


Pre-Observation 


22.00 4.86 


21.25 6.65 


0.22 


NS 


Post-Observation 


30.71 3.86 


31:00 5.60 


, 0.10 


".NS 



I 

I 

\ 

\ 

I 
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Statistics from the pre- and post-observations of recommended behaviors 
by teachers within the two project schools are displayed in Table 13. Taking 
both schools' teachers together, the- mean number of correct implementations 
of recommended teaching behaviors for all variables (N=45) was 21.73 while 
the post-observation correct implementations mean was 30.82, an increase of 
nine correct -implementations. The increase between pre-observation correct 
implementations and the post-observation correct- implementations was signifi- 
cant at the .01 level. When the 45 teaching behavior variables were divided 
into the three Stall ings classification types and the pre- and post-observa- 
tions compared within the two schools' faculties, it can be seen that signi- 
ficant differences were observed at the .05 level for the group of 28 inter- 
action instructional variables and .at the ,01 level for the nine off-task 
behaviors. There was no significant difference in the pre-post-observations 
■of correct implementations of recommended teaching behaviors in the eight 
non-interaction instructional behaviors. When the 45 teacher variables were 
compared by the three project-determined categories, it can be seen that the 
v 17 classroom management variables produced a significant - ( .001 level) differ- 
ence between" the pre-' and post-observations of correct implementations of 
recommended teaching behaviors. The mean number of correct implementations 
increased from 7.82 at pre-observation time to 12.91 at post-observation time. 

Table 13 also displays the pre-post-observations of correct implementa- 
tions of « recommended teaching behaviors for all 45 variables by each school's 
faculty. Here it is shown that the mean number of correct implementations for 
School A teachers increased from 22.00 to 30.71 which was significant at the 
.05 level. Table 13 also shows that the mean number, of correct implementations 
for School B teachers increased from 21.25 to 31.00. However, due to the small 
N even this large increase was not significant (t^f 3 ) = 3 -18). 
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.Statistics from Pre- and Post-Observations of Correct Implementations 
of Recommended Teaching Behaviors Within Schools 



Group 

• 


Pre -Obs 




1 

Post- 


-Obs. 


t- 


Sig. 


Mean 


SD 


Mean 


SD 


Value 


Level 


Both Hi qh^Sjthool s 
(N=ll) * 


« 




V 






- 


Al 1 Teaching .Vari- 
ables (45) 


21.73 


5.26 


30.82 


4.29 


3.99 

i 


.01 


Variables Types ^Stal lings grouping) 






a 


- 


Interaction Instruc- 
tion Variables (28) 


11.82 


4.26 


16.36 


3.47 


2.62 


.05 


Non-interaction 
Instructional 
Variables (8) 


4.09 


1.14 • 


4.82 


« 

1.17 


1.99 


Si 

NS 


Off-Task Varia- 
bles (9) 


6^.Q9 


1.64 


7.73 


0.90 


3.63 


.01 


Variable Categories (Project Grouping) 










Instruction (20) 


9.91 


3.27 - 


12.64 


2.34 


2.11 


NS 


Classroom Manage- 
ment (17) 


Q 

7.82 


1.94 


• 12.91 


1 .45 


6.15 


.001 


Feedback and Disci- 
pline (8) 


4.27 


1.62 


5.27 


r 

1.85 


1.82 


NS 


School A, (N=7) 

All Teachinq Varia- 
bles (45) 

4 


22.00 


4.86 


30.71 


3.86 


3.15 


.05 


School B (N=4) 














All Teaching Varia- 
bles (45)' 


21-25 


6.65 - 


31.00 


5.60 


1.97 


NS 




J 




Teaching Behaviors Change Index 

. The Stallings Classroom Management Staff Development Project provides a 
unique opportunity to study teachers 1 classroom behavior changes. Because 
pre- and post-observations of classroom behaviors are involved; because 
5£search-based recommendations of changes in specific classroom teaching 
behaviors are made; and because the project teachers received ideas, support, 
and resources to help in the behavior change process, the Stallings staff 
development program provides 9 unique opportunity to describe and assess the 
actual t amount of teaching behavior changes. \ 

The Teaching Behaviors Change Index is >an original measure conceived and 
reported for the first time in this report. The Teaching Behaviors Change 
Index (TBCI) is an interval level figure expressing the amount of change in 
specific teaching behaviors before and after tfeacher training workshops. 
Literally, TBCI is the number of correct implementations of specific teaching 
behaviors after the teacher training workshops minus the number of correct 
implementations of specific teaching behaviors before the teacher training 
workshops divided by the number of variables (teaching behaviors) observed. < 
The formula for the TBCI is: 

"a - "b • 

' ' ' 

where: \ 

I = Teaching Behaviors Change Index 

N A = Number of correct implementations of teaching behaviors 
after teacher training workshops 

% = Number of correct implementations ,pf teaching behaviors 
before teacher trailing workshops 

V = Number of teaching behaviors (variables) studied 
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Examp 



es of the TBCI are: 



A 




N R 

D _ 


HD " U _ 




V 




45 - 


9 








_ A 




N R 


QQ 1 7 




V 




45- 


> N-j 
A 




B _ 


26 - 16 




V 




45 . 


. N A 




V- 


10 - 10 




V 




;<• 45. ' 


N A 




N B_ 


22 - 45 




V 




45 


N A 




N B_ 


0 - 45 _ 




V 




■ *,45 



= +1.00 



+0.47 



+0.22 



■0.51 



Possible TBCI values range from -1.00 to +1.00. This continuum of TBCI 
values should be viewed as a range of values from low (least desirable) to 
high (most desirable). In this manner, then, the TBCI values are not similar 
to correlation values where a. perfect negative correlation may be- desirable. 
It is, however, too soon to assign descriptors to any of the various obtained 
TBCI values. 

Table 14 displays the Teaching Behaviors Change Index values for the 
eleven Putnam County demonstration project teachers. The TBCI values ranged 
from a high of +0.4g to a low of -0.07. The mean TBCI value was +0.21 "with 
a standard deviation of 0.17. 

One particular TBCI value in Table 14 merits further discussion. The 
demonstration project teachers were instructed specifically not to change or 
alter their teaching schedules during pre- and post-intervention observa- 
tions. As a consequence of adhering to this request, one teacher in the 
project was observed at post-intervention time in a series of prearranged 



Table 14 



Teaching Behaviors Change Index, Values 
for Demonstration Project Teachers 



Tparhpr Dp^ianation 




N B 


Teaching Behaviors 
Phanap Index Value* 


Teacher A 


35 


19 


+0.36 


v. ^ Teacher B 


27 


30 " 


-0.07 


Teacher C 


26 


16 


+0.22 


Teacher D 


38 


19 


+0.42 


Teacher E 


28 


cb 


- In (ii i ' 


Teacher F 


,33 


28 


+0.11 


Teacher G 


30 


19 


+0.24 


Teacher H 


38 


16 


+0.49 


Teacher I 


. 25 


20 


+0.11 


Teacher J 


32 


* 18 


+0.31 


Teacher K 


31 


27 


+0.09 



♦Possible values range. from -1.00 to +1.00: See text for the formula and 
its explanation. 



instructional activities which did not involve interactive instruction-- 
although they did involve instruction. Thus, by following the project's 
instructions regarding lesson planning, his/her classroom behaviors were 
observed during three days of planned primarily noninteractive instruction. 
Since the noninteractive instructional activities concluded on the third 
observation day, there was no chance to include mor,e typical instructional 
activities at post-observation time. He/she knew about and realized the 
probable outcomes in terms of post-oBsfervation results regarding the low 
number of correct implementations. As expected, this person is the only 
teacher who obtained' a negative TBCI value. This fact had a negative effect 
upon the teacher and it was expressed openly during the taped interview. 
This^teacher was positive about the overall project but was influenced 
negatively at the very end by post-observation scheduling procedures. The 
teacher wanted to show changes in behaviors, felt he/she could demonstrate 
changes, but was thwarted by the scheduling of the post-observations in rela- 
tion to his or her lesson plans. 

Principals', Teacher Tra'jners 1 , and 
Superintendent's Reactions" 

Both principals of the demonstration schools were interviewed at the < j 
completion of the project in order to obtain their reactions. The princi- 
pals offered several responses when asked to name any positive features of 
the demonstration project. Both principals said that the opportunity for 
teachers to discover exactly what their teachincj^ehaviors were by objective 
coders was a positive feature. The selection of observers from the substi- 
tute teacher list was another positive feature they mentioned.. One principal 
opined that his involved teachers "gained in self-security and self-confi- 
dence." In response to the question of the effects of their involvement on 
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the teacher, each principal replied differently. . One principal stated that 
the teachers were more "close knit" as a 'group now: a good outcome. The 
other high school principal felt that the project teachers were more aware 
„now of their strong and weak points in teaching and, thus, had a good chance 
to become better teachers. . Asked about possible side effects of the demon-' 
stration project, one principal felt that the aspect of "outside" persons 
from other educational agencies observing and interacting with his or hei* 
staff was a positive side effect. Both principals said they had no problems 
in implementing the staff development project. Both principals said they 

> 

felt that the project had the strong support of their supervisors (the - 
central office staff). Asked if they would 1 ike more detailed information 
about the project, both principals replied in the affirmative. When asked 
if they were in favor of the continuation of the/cl assroom management staff 
development project next year, both princj^als replied yes, but with a condi- 
tion. One principal's condition was that the' other parts of the comprehen- 
sive instructional improvement program (Higginbotham, 1981) should begin and 
the other principal felt that the' same leader (teacher .trainer) should manage 
the project. 

Using basically the same interview schedule, the teacher trainer was 
interviewed by the evaluator. Asked to state any positive features of the 
demonstration project, the teacher trainer named three: (1) "building 
cohesiveness, comaraderie among teachers within schools," (2) "broadening of 
teachers' perceptions of themselves personally and professionally," and (3) 
"expanding teachers' knowledge about classroom and instructional management 
and their relationship to student achievement." Asked to state what the 

o 

effect of the project had been on the teachers, the teacher trainer responded 
that the project "definitely has opened some eyes and some minds." The 



te'acher trainer- stated that the project haUi m Pa cted bot b other teachers and 

students. Other teachers heard about the demonstration project and became 

— ~\ \ 
interested in it. Some of the project teachers to.Ttkthe teacher trainer 

that the'ir students noticed a difference in their teaching-and that they, 

"enjoyed the 'new' teachers." Regarding the possible continuation of the 

project, the teacher trainer was very enthusiastic, saying: \^ 

I'.d like to expand the sessions tckwork with new (and more) 
- teachers, yet also have the opportunity to have some> 
follow-up, informal sessions with last year's group Of 
; ' teachers as, well as collect follow-up [observational] data 
» • on them. 

The teacher trainer was very candid with opinions "of are^sr where the 

project which could be improved. The teacher trainer made seven specific 

recommendations for improvement: 

1. Use volunteer .teachers in the future. 

.2. Schedule the teacher training workshops every other 

week instead of weekly. w 

3. Make provisions for the teacher trainer to have more 
contact with* the project teachers in their classrooms,' 

4. Distribute the teacher profile charts at the second 
workshop. 

5. Expand the data > col lection process from three to five 
days. 

6. Eventually expand the project to include teachers from 
other disciplines and grade levels. 

7. Secure the optical scanning and the profile chart 
computer programs. •» 

The Superintendent of Putnam County Schools was interviewed at the con- 
clusion of the project using nearly the same interview questions. Asked 
about the possible positive aspects of the demonstration, the Superintendent 
offered two responses. First, the Superintendent said that the project 
"...brought to Putnam County teachers a validated, systematic 'approach of 



63 



classroom observations by which teachers can analyze theirteaching behaviors 
and interactions with students." Second, the project seemed to have fostered - 
atgenerally favorable attitude among many teachers and had spurred, comments 
toward school and classroom management. .Asked to comment on the effects on 
the project teachers, the Superintendent replied that he felt that there was 
a "greater realization and/or renewal of the vitally important role that 
teachers have in the educational process." Side effects of the project 
include inquiries from other teachers about the project's processes and.- 
expansion plans which could involve more teachers. 'Regarding support from , 
superiors, the Superindentent pointed to the fact that the project is an , 
important part of a comprehensive' instructional improvement program (Higgin- 
botham,. 1981). which has the official endorsement of the Putnam County Board 
of Education. Two concerns' were expressed when asked about any problems in 
the implementation of the project. First, the Superintendent mentioned the 
difficulty of finding time in his work schedule to devote to "the project 
(although there was no lack of interest). Second, the Superintendent had 
heard some expressions of concern apout the way administrators will vieW the 
process-as professional .growth or as teacher evaluation. The Superintendent 
felt -strongly that the project should continue and that it should be adjusted/ 
refined with information "that will enable the process to be tailored to the 
needs of Putnam County Schools, its administrators, teachers, and especially, 
its students." 
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CHAPTER VI 

\ 

Conclusions and Recommendations p %' 

Process evaluation results were presented in chapter four. Product eval- 
uation results were^presented in chapter five. This chapter will presrent the 
conclusions and recommendations from the evaluation of the Stallings Class- 
room Management Staff Development Demonstration Project in Putnam County, 
West Virgina. 

« 

Conclusions 1 ^ 

This evaluation study was designed and conducted to assess the implemen- 
tation of a specific staff development demonstration project. Based on the 
data collected and reported in .the previous chapters, certain conclusions can 
be drawn. These conclusions shall be presented in this sectfon. 

Overall, r'esults of this evaluation show that the Stallings Classroom 
Management Staff Development Demonstration Project implemented in Putnam 
County Schools in 1980-1981 was a success. Findings from five out of six 
data collection instruments/techniques lead to the conclusion of the project 
being a success. The demonstration, .project teachers' reactions/feelings 
advanced from a point just above neutral to a point one full point higher 
(on a six-point continuum) at the conclusion of the project. During tqped 
interviews, project teachers made two and one half times as many positive 
evaluative comments about the project as negative evaluative comments. There, 
was a significant increase in the project teachers' expressions of positive s 
responsibility for student achievement, taking into account pretest scores. 
Levels of Use ratings confirmed that there were eight innovation users and 
just three non-users, among the eleven teachers. Finally, data analysis 
showed a significant increase in the number of correct implementations of 



teacher behaviors change (as defined by the program) from 4 pre-intervent ion 
observation time to post-intervention^observation time.. 

Each data collect ion. procedure provides specific findings to the overall 
conclusion. First, the demonstration project teachers 1 reactions/feelings 
regarding the project events moved upward a^ the events unfolded. Starting 
one half point above neutral (on a "+3" to "-3" continuum), the teachers 1 * 
reactions/feelings mean score dipped to SHow pofnt just after they received 
'the teacher behavior profile charts, -then moved' upward ta the value of +1.6. 
This movement on this instrument is an indication- of project success. 

Second, the demonstration project teachers expressed high concerns about 
the project at its conclusion. The intensity level. of these concerns was 
high,' although not unexpe^ed for a brand new innovation regarding their 
.teaching practices'. Combining the concern scores from the two demonstrate 
site faculties yielded the most 1 ' intense concern as Informational. The 
teachers desiWd more information about the innovation. There was a svgnifi 
cant difference across school faculties regarding the Personal stage with* - 
*one school facul typing particularly concerned about their professional 
status or rale change in the organization in relation to the innovation. 

Third, in' evaluating the project after it concluded, participating 
teachers overwhelmingly judged it positively according to their evaluation/ 
assessment comments. .Analysis o of. 685 individual evaluation/assessment 
comments revealed two and one half times as many positive comments in 
comparison to negative comments'. Of all the evaluative comments, just about 
one half- pertained to the teacher training workshops and there were more 1 

• favorable comments than negative; ones on this-topic. The remaining ^ m 

' * ■ ' ' \ ■ _ . ^- --• - " ~ *~ ** *~ 

evaluative comments were ; divided among seven other catejorifis, 
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Fourth, the- demonstration project teachers ' self-expressed positive 
responsibility for student achievement scores increased significantly from 
prestest to postte$t administrations of the instrument. , In the absence of a 
control or comparison group design it is not possible to conclude that the 
project activities caused the increase in perceTve3>espo'nsibfl ity for 
student achievement; however, the pretest-posttest increase is a fact, even 
taking into account the significant differences in pretest scores across the 
two schools' faculties. 

Fifth, there was a high number of innovation users ^pr this demonstration 
project. Th t e Levels of Use Ratings showed that there were eight (73%) users 
'and just three (27%) non-users.of the innovation. Given/that: (1) the 
teachers did not volunteer themselves to be participants in the project, m 
(2) the complexity of the innovation, (3) the timing of the project in the 
school year, and (4) the content of the .project (changing teachers' class- 
room behaviors) the number of users. was high. 

Sixth, the staff development project had, a significant impact on changing 
teachers' behaviors. The mean number of" correct implementations (as defined 
fey the program) of recommended teaching behaviors increased from 22 at pre- 
observation time ."to 31 at ^post-observation time. This, pre-post difference 
was significant.! F.urther 1 , 'the 45 teaching behavior variables can be broken 
down into logical groupings and the location. of the significant differences 
can be pinpointed further. • • 

Finally, the Teaching Behaviors Change Index is an paginal interval 
level measure of thefparticipating teachers' changes, in classroom behaviors. 
- Based ^n-syst«mati^,^b^eetwe^ob5ervations at pre- and post-intervention 
periods and' on specific, research-based recommendations -for changes 1 n _ _ ^ 
teaching behaviors, the Teaching Behaviors Change Incjex is a measure which . 
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communicates the level of performance for teachers involved in this particu- 
lar staff development program. It can be used as a commutation and compar- 
, ison measure across several implementations of this staff development program 
in one' locale, but further, it can serve to describe and assess implementa- 
tions of this staff development- program. across several different project 
installation sites. Most important, over time and'with the collection of 
valid and reliable background and environmental variables relating to 
teachers' jobs, the Teaching' Behaviors Change Index can be used, as the 
dependent variable in studies to discover what independent variab-les impact 
oh it . Thus, the knowledqe base of changing teachers - cl assroom behaviors 
may be increased greatly. 

Recommendations 

in view of the- findings and iconclusions of this evaluation study and 
based on nearly a year's work with the demonstration project, certain recom- 
mendations are offered to the administrator in charge and to other readers. 

Several recommendations dea] with continuation of the Stallings Classroom 
Management Staff Development Program. If Putnam Qounty Schools' administra- 
tors are look-ing for an objective, research-based staff development program 
that can demonstrate positive changes in classroom teaching behaviors by 
+ Putnam County teachers, then they h^ave found -it .in the Stallings program. 
Assuming no other constraints, there was nothing discovered in the evaluation 
of the demonstration project to discourage its continued use and, resources 
■ permitting, it* expansion. Some refinements in the model to make it more 
\ relevant to Putnam County teachers are'suggested on the following pages, but 
1 these are considered slight adjustments and not major modifications. 
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Several specific recommendations deal with the processes of the Stall ings 
program. It is recommended that the classroom observers/coders continue to 
be selected from the Putnam County Substitute Teacher List by a committee of 

* « ** * ' 

Putnam County teachers and administrators. It is recommended strongly that 
"{he classroom observations be expanded' to five class periods per observation 
set (Fall, Winter, Spring). Also, it is recommended strongly that these five 
classroom observations be spread over a two week time period to allow for 
individual instructional scheduling possibilities. Next, it -is> recommended 
just as strongly that the. teacher behavior profile* charts not be distributed 
and discussed until the second' teacher training workshop' . Concomitantly, it 
is recommended that the total number of teacher training workshops °e 
expanded to six and that the first one shculd deal with and prepare the 
teachers for the receipt of their teacher /behavior profile charts. All other 
aspects of the teacher training workshop/ can remain the same as implemented 
in the demonstration project/, based on this evaluation. ^ /~\ 

Informational needs is another area for recommendations. There \\ an I 
intense concern for more information about the classroom-management staff 
developmenJ^project by Putnam County teachers and administrators. It is 
recommended that a planned and coordinated information program be designed 
and implemented. It is recommended, for example, that if volunteer teachers 
will be involved in the next phase, a* teacher recruitment brochure or flyer 
be prepared which will communicate basic information about the staff develop- 
ment project. It is recommended that this brochure or flyer be followed with 
personal awareness sessions for new project teacher recruits including the. 
Superintendent, the teacher trainer, an observer/coder, and a participating 
teacher from the first year's installation. But the informational needs 
shouldn't end at this stage. "It is recommended that many more mechanisms 



for "getting the word out" be put in place. These could include: (1) a- 
distinctive, attractive, and colorful logo identifying the program arid/or 
project^ (2) insertsUo the school system's newspaper; (3) special bulletins 1 
to teachers; and (4) memos or letters to teacher and principals reinfprcing 
the Intent to continue with the thrust begun in the 1980-1981 schdol' year. 
Other informational possibilities should be studied. 

Independence and self-sufficiency are the targets for other recommenda- 
tions. If Putnam County Schools continues the classroom management staff 
development program, then several important steps leading to their self- 
sufficiency with the system are recommended. A Putnam County "apprentice" 
in the Stallings system should be identified and trained by the Stallings-, 
trained apprentice as soon as possible. Without a locally-supported. 
Stallings "apprentice," the domino effect may never begin in Putnam County 
Schools and the school system will be dependent always on an outside agency 
for the teacher trainer. The observation system's optical scanning program 
should be purchased to net- only reduce data processing costs, but more 
importantly, to speed-up the turnaround time between observations and the 
production of the teacher behavior profile charts. Similarly, the computer 
program(s) which transform tire raw data into the teacher behavior profile 
charts should be purchased, if possible, from the Teaching and Learning 
Institute. These steps identified above will ensure a large measure of 
independence and self-sufficiency by the Putnam Cpunty Schools and, in the 
long run, save considerable amodnts of money for the school system. 

Finally, several recommen'dations foV research, development^ and evalua- 
tion are offered. A natural recommendation would be to monitor continuously 
the implementation of the Stallings staff development. program in Putnam 
County Schools, At a minimum, a design similar to the one. used in this fc 

$ -- { ' h. t «. *" 
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report may be used, although the inclusion of a comparison (non-project) 
grouprbf- teachers would increase t£e Internal and external validity of the 
.evaluation results. rmprovementV-io^instrumentatibn should be attempted in 
future evaluations. The original group of eleven demonstration project^ 
teachers should be studied over time including, if possible, continued 
observation of their classrooms and ^refresher" training workshops to assess 
possible long term effects of their involvement in the. demonstration project. 
Putnam County Schools, with outside assistance as necessary and appropriate, 
•should begin to develop its own data' base for the teacher behavior profile 
charts' recommendations of classroom behavior changes. This could include, 
for example, a change to criteriorv-ref erenced measures of student achieve- 
ment outcomes. Over time, various criterion-referenced data bases could be 
built into the system to accommodate vai/ious content ar^as. Continued jjse. 
of the Teaching Behaviors Change Indexes encouraged. Communication of this 
measure between and among others\jsing the Stal lings staff development 
program is recommended. The first goal of this network of index users could 
be the initial assignment of descriptors to various attained values of the 
index. This could assist in the development of a common metric for 
describing and assessing the actual amount of teaching behavior changes. 
Associated with the previous recommendation would be exploratory study of 
the Teaching Behaviors Change Index as the dependent variable in staff 
development/teacher effects research. .Last, results of this evaluation 
study should be communicated to other educational personnel through this 
repdrU the executive summary,' articles, research papers, and/or presen- 
tationi'at local, s t at e^ region^ and. national roeeti-rigs and- conferences. 
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Appendix A: 
Reaction Survey 
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'Putnam County Classroom Management/Teacher Effectiveness 

Demonstration Project 

Reaction Survey 0 * 
, June 8, 1981 

• • * 

Directions: /In prose form, please provide us with your reactions/feelings 
as this demonstration project unfolded. You may want to use the following 
•outline to organize your narrative. 



1 . Before the teacher awareness meeting with Ken Higgiribotham, Joe Basile, 
Debra Sullivan: 3 



/ 



2. After the teacher awarentfts meeting : 



3. Before the first set of observations: 



4. After' the first set of observations: 



5 . After the first teacher .training session : 



6. After the third teacher training session : 



7. After the final teacher training session : 



8. Before the second set of observations: 



9. After the second set of observations: 



10. Before receiving your f inaJ profile : 



11. After receiving your final profile : 



12. Today: (6/8/81) : 



9. 
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Appendix B: 

Citation Form: The Standards for Educational 
Programs, Products, and Materials 



Citation *om* 

The Standards for Equations of Educational Programs, ftp/eta, andMatwriah juided the development of this (chtck one ) 
request for evaluation plan/detign/proposa! 

eve tuition i plen/deeign/propou! 78 
* •valuation contract 
^ evaluation report ^ 

ilher 

To interpret the informetion provided on this form, the reader needs to refer to the full text of the standards as they appear in 
Joint Committee on Standards for Educational Eveluetion, Standards forj¥aluations of Educational Programs, Projacts, and 
Matariati New York: McGrew Hill. 1980. \ 

The Standard were consulted end used at indicated in the table below (check ai appropriate) : 

The Stenderd was The Standard was The Stenderd was 
deemed applicable deemed applicable not deemed eppli- 
end to the extent 
feasible wes taken 
Descriptor # into account 



At 
A2 



Audience Identification 



Evaluetpr Credibility 



XXXX 



XXXX 



The Stenderd wes 
deemed applicable 
itMit could not be 
taken into account 



cable 



Exception wes 
taken to the 4 
! Standard 



A3 - Information Scope end Selection 
/*A4 Valuationel Iriterpretetion 



XXXX 



XXXX 



A5 
A6 
A7 
AB 
B1 
B2 



Report ClerftV 



XXXX 



WeporfrDiueminetion 
Report Timeliness 



XXXX 



XXXX 



Evaluetion Impect 



XXXX 



Practical Procedures 



XXXX 



Political Viability 



XXXX 



,B3 Cost Effectiveness 



XXXX 



CI Forme! Obligation 



XXXX 



C2 
C3 
C4 
C5 



Conflict of Interest 

Full and Frank Disclosure 



XXXX 



XXXX 



Public's Right to Know 



XXXX 



Rights of Human Subjects 



XXXX * 



C6 Humen Interactions 



* XXXX 



C7 
C8 
D1 
D2 
D3 
D4 



Balanced Reporting 



XXXX 



Fiscal Responsibility 



XXXX 



Object Identification 



XXXX 



Context Analysis 



XXXX 



$5 



Described Purposes and Procedures 



XXXX 



Definable Information Source! 



XXXX 



D5 Valid Measurement 




XXXX 



D6 
D7 



Reliable Measurement 
Systematic Data Control 



XXXX 



XXXX" 



08 Analysis of Quantitative Information 

09 Analysis of Qualitative Information 



XXXX 



y xxxx 



010 Justified Conclusions' 



5 



XXXX 



011 Objective Reporting 



X.XXX 
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